>>> kops: Building testing/kops 1.23.0-r2 (using abuild 3.9.0-r0) started Thu, 12 May 2022 07:40:01 +0000 >>> kops: Checking sanity of /home/buildozer/aports/testing/kops/APKBUILD... >>> kops: Analyzing dependencies... >>> kops: Installing for build: build-base kubernetes go (1/3) Installing kubernetes (1.24.0-r2) (2/3) Installing go (1.18.2-r0) (3/3) Installing .makedepends-kops (20220512.074002) Executing busybox-1.35.0-r13.trigger OK: 711 MiB in 101 packages >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.23.0.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 28.7M 100 28.7M 0 0 64.6M 0 --:--:-- --:--:-- --:--:-- 64.5M >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.23.0.tar.gz >>> kops: Checking sha512sums... kops-1.23.0.tar.gz: OK >>> kops: Unpacking /var/cache/distfiles/edge/kops-1.23.0.tar.gz... go: downloading github.com/aws/aws-sdk-go v1.43.11 go: downloading k8s.io/klog/v2 v2.30.0 go: downloading k8s.io/apimachinery v0.23.1 go: downloading k8s.io/api v0.23.1 go: downloading k8s.io/client-go v0.23.1 go: downloading sigs.k8s.io/controller-runtime v0.11.0 go: downloading sigs.k8s.io/yaml v1.3.0 go: downloading github.com/blang/semver/v4 v4.0.0 go: downloading github.com/jetstack/cert-manager v1.6.1 go: downloading github.com/spf13/cobra v1.2.1 go: downloading github.com/spf13/viper v1.8.1 go: downloading k8s.io/cli-runtime v0.23.1 go: downloading github.com/blang/semver v3.5.1+incompatible go: downloading k8s.io/klog v1.0.0 go: downloading github.com/google/uuid v1.3.0 go: downloading github.com/gophercloud/gophercloud v0.24.0 go: downloading google.golang.org/api v0.57.0 go: downloading github.com/go-logr/logr v1.2.0 go: downloading github.com/Azure/azure-sdk-for-go v56.2.0+incompatible go: downloading github.com/digitalocean/godo v1.65.0 go: downloading github.com/Azure/go-autorest/autorest/azure/auth v0.5.7 go: downloading golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8 go: downloading github.com/Azure/go-autorest/autorest v0.11.19 go: downloading cloud.google.com/go v0.97.0 go: downloading github.com/Azure/go-autorest v14.2.0+incompatible go: downloading k8s.io/legacy-cloud-providers v0.23.1 go: downloading github.com/aws/amazon-ec2-instance-selector/v2 v2.0.2 go: downloading github.com/google/go-containerregistry v0.7.0 go: downloading github.com/spf13/pflag v1.0.5 go: downloading golang.org/x/crypto v0.0.0-20211215153901-e495a2d5b3d3 go: downloading helm.sh/helm/v3 v3.7.1 go: downloading k8s.io/kubectl v0.23.1 go: downloading gopkg.in/yaml.v2 v2.4.0 go: downloading github.com/davecgh/go-spew v1.1.1 go: downloading github.com/hashicorp/hcl v1.0.0 go: downloading github.com/fsnotify/fsnotify v1.5.1 go: downloading github.com/magiconair/properties v1.8.5 go: downloading github.com/mitchellh/mapstructure v1.4.1 go: downloading github.com/pelletier/go-toml v1.9.3 go: downloading github.com/spf13/cast v1.3.1 go: downloading github.com/spf13/afero v1.6.0 go: downloading github.com/spf13/jwalterweatherman v1.1.0 go: downloading github.com/subosito/gotenv v1.2.0 go: downloading gopkg.in/ini.v1 v1.62.0 go: downloading github.com/stretchr/testify v1.7.0 go: downloading github.com/inconshreveable/mousetrap v1.0.0 go: downloading github.com/evanphx/json-patch v5.6.0+incompatible go: downloading github.com/prometheus/client_golang v1.11.0 go: downloading k8s.io/utils v0.0.0-20210930125809-cb0fa318a74b go: downloading k8s.io/component-base v0.23.1 go: downloading github.com/gogo/protobuf v1.3.2 go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.2.0 go: downloading github.com/google/gofuzz v1.2.0 go: downloading gopkg.in/gcfg.v1 v1.2.3 go: downloading github.com/google/go-cmp v0.5.6 go: downloading gopkg.in/inf.v0 v0.9.1 go: downloading github.com/apparentlymart/go-cidr v1.1.0 go: downloading github.com/googleapis/gnostic v0.5.5 go: downloading github.com/MakeNowJust/heredoc/v2 v2.0.1 go: downloading github.com/imdario/mergo v0.3.12 go: downloading golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 go: downloading github.com/Azure/azure-pipeline-go v0.2.3 go: downloading github.com/Azure/azure-storage-blob-go v0.13.0 go: downloading github.com/denverdino/aliyungo v0.0.0-20210425065611-55bee4942cba go: downloading github.com/go-ini/ini v1.62.0 go: downloading github.com/hashicorp/vault/api v1.1.1 go: downloading github.com/pkg/sftp v1.13.0 go: downloading golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd go: downloading github.com/MakeNowJust/heredoc v0.0.0-20170808103936-bb23615498cd go: downloading github.com/google/go-querystring v1.0.0 go: downloading k8s.io/kubelet v0.23.1 go: downloading golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e go: downloading k8s.io/mount-utils v0.23.1 go: downloading github.com/sergi/go-diff v1.2.0 go: downloading gopkg.in/square/go-jose.v2 v2.5.1 go: downloading github.com/Azure/go-autorest/autorest/to v0.4.0 go: downloading github.com/Azure/go-autorest/autorest/adal v0.9.14 go: downloading github.com/Azure/go-autorest/autorest/azure/cli v0.4.2 go: downloading github.com/Azure/go-autorest/logger v0.2.1 go: downloading github.com/dimchansky/utfbom v1.1.1 go: downloading github.com/spotinst/spotinst-sdk-go v1.85.0 go: downloading github.com/Masterminds/sprig/v3 v3.2.2 go: downloading github.com/jacksontj/memberlistmesh v0.0.0-20190905163944-93462b9d2bb7 go: downloading github.com/weaveworks/mesh v0.0.0-20191105120815-58dbcc3e8e63 go: downloading github.com/aliyun/alibaba-cloud-sdk-go v1.61.1059 go: downloading github.com/google/go-tpm-tools v0.3.0-beta1 go: downloading github.com/google/go-tpm v0.3.2 go: downloading github.com/hashicorp/hcl/v2 v2.10.1 go: downloading github.com/zclconf/go-cty v1.8.2 go: downloading k8s.io/gengo v0.0.0-20210813121822-485abfe95c7c go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.1 go: downloading k8s.io/cloud-provider v0.23.1 go: downloading k8s.io/csi-translation-lib v0.23.1 go: downloading golang.org/x/sync v0.0.0-20210220032951-036812b2e83c go: downloading sigs.k8s.io/json v0.0.0-20211020170558-c049b76a60c6 go: downloading github.com/golang/protobuf v1.5.2 go: downloading github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7 go: downloading github.com/peterbourgon/diskv v2.0.1+incompatible go: downloading google.golang.org/protobuf v1.27.1 go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de go: downloading golang.org/x/text v0.3.7 go: downloading sigs.k8s.io/kustomize/api v0.10.1 go: downloading sigs.k8s.io/kustomize/kyaml v0.13.0 go: downloading k8s.io/kube-openapi v0.0.0-20211115234752-e816edb12b65 go: downloading gomodules.xyz/jsonpatch/v2 v2.2.0 go: downloading k8s.io/apiextensions-apiserver v0.23.1 go: downloading google.golang.org/grpc v1.42.0 go: downloading google.golang.org/appengine v1.6.7 go: downloading github.com/GoogleCloudPlatform/k8s-cloud-provider v1.16.1-0.20210702024009-ea6160c1d0e3 go: downloading github.com/onsi/ginkgo v1.16.5 go: downloading github.com/onsi/gomega v1.17.0 go: downloading go.uber.org/goleak v1.1.12 go: downloading github.com/go-logr/zapr v1.2.0 go: downloading go.uber.org/zap v1.19.1 go: downloading gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c go: downloading golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac go: downloading gopkg.in/warnings.v0 v0.1.2 go: downloading github.com/smartystreets/goconvey v1.6.4 go: downloading github.com/pmezard/go-difflib v1.0.0 go: downloading gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b go: downloading github.com/pkg/errors v0.9.1 go: downloading github.com/mattn/go-ieproxy v0.0.1 go: downloading github.com/docker/cli v20.10.10+incompatible go: downloading github.com/prometheus/client_model v0.2.0 go: downloading github.com/prometheus/common v0.28.0 go: downloading github.com/cenkalti/backoff/v3 v3.0.0 go: downloading github.com/hashicorp/errwrap v1.0.0 go: downloading github.com/hashicorp/go-cleanhttp v0.5.1 go: downloading github.com/hashicorp/go-multierror v1.1.0 go: downloading github.com/hashicorp/go-retryablehttp v0.6.6 go: downloading github.com/hashicorp/go-rootcerts v1.0.2 go: downloading github.com/hashicorp/vault/sdk v0.2.1 go: downloading github.com/go-test/deep v1.0.3 go: downloading github.com/hashicorp/go-hclog v0.16.1 go: downloading github.com/beorn7/perks v1.0.1 go: downloading github.com/cespare/xxhash/v2 v2.1.1 go: downloading github.com/prometheus/procfs v0.6.0 go: downloading github.com/Azure/go-autorest/tracing v0.6.0 go: downloading github.com/Azure/go-autorest/autorest/mocks v0.4.1 go: downloading github.com/json-iterator/go v1.1.12 go: downloading github.com/Azure/go-autorest/autorest/date v0.3.0 go: downloading github.com/mitchellh/go-homedir v1.1.0 go: downloading github.com/kr/fs v0.1.0 go: downloading github.com/form3tech-oss/jwt-go v3.2.3+incompatible go: downloading github.com/Masterminds/goutils v1.1.1 go: downloading github.com/Masterminds/semver/v3 v3.1.1 go: downloading github.com/huandu/xstrings v1.3.2 go: downloading github.com/mitchellh/copystructure v1.1.1 go: downloading github.com/shopspring/decimal v1.2.0 go: downloading github.com/hashicorp/go-sockaddr v1.0.2 go: downloading github.com/hashicorp/memberlist v0.1.4 go: downloading github.com/oklog/ulid v1.3.1 go: downloading golang.org/x/tools v0.1.8 go: downloading github.com/googleapis/gax-go/v2 v2.1.0 go: downloading go.opencensus.io v0.23.0 go: downloading github.com/ghodss/yaml v1.0.0 go: downloading github.com/russross/blackfriday/v2 v2.1.0 go: downloading github.com/stretchr/objx v0.2.0 go: downloading github.com/docker/distribution v2.7.1+incompatible go: downloading golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 go: downloading github.com/containerd/stargz-snapshotter/estargz v0.10.0 go: downloading github.com/jmespath/go-jmespath v0.4.0 go: downloading github.com/apparentlymart/go-textseg/v13 v13.0.0 go: downloading github.com/kylelemons/godebug v1.1.0 go: downloading github.com/agext/levenshtein v1.2.1 go: downloading github.com/kr/pretty v0.3.0 go: downloading github.com/mitchellh/go-wordwrap v1.0.0 go: downloading github.com/google/btree v1.0.1 go: downloading github.com/chai2010/gettext-go v0.0.0-20160711120539-c6fed771bfd5 go: downloading github.com/russross/blackfriday v1.5.2 go: downloading github.com/google/go-attestation v0.3.2 go: downloading go.uber.org/atomic v1.7.0 go: downloading go.uber.org/multierr v1.6.0 go: downloading github.com/fatih/color v1.12.0 go: downloading github.com/mattn/go-colorable v0.1.8 go: downloading github.com/mattn/go-isatty v0.0.12 go: downloading github.com/mitchellh/reflectwalk v1.0.1 go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd go: downloading github.com/modern-go/reflect2 v1.0.2 go: downloading google.golang.org/genproto v0.0.0-20211111162719-482062a4217b go: downloading github.com/opencontainers/image-spec v1.0.2-0.20210730191737-8e42a01fb1b7 go: downloading github.com/moby/term v0.0.0-20210610120745-9d4ed1856297 go: downloading github.com/kr/text v0.2.0 go: downloading github.com/rogpeppe/go-internal v1.6.1 go: downloading github.com/nxadm/tail v1.4.8 go: downloading github.com/containerd/containerd v1.5.7 go: downloading github.com/sirupsen/logrus v1.8.1 go: downloading oras.land/oras-go v0.4.0 go: downloading github.com/distribution/distribution/v3 v3.0.0-20210804104954-38ab4c606ee3 go: downloading github.com/phayes/freeport v0.0.0-20180830031419-95f893ade6f2 go: downloading github.com/klauspost/compress v1.13.6 go: downloading github.com/opencontainers/go-digest v1.0.0 go: downloading github.com/vbatts/tar-split v0.11.2 go: downloading github.com/ryanuber/go-glob v1.0.0 go: downloading github.com/golang/snappy v0.0.3 go: downloading github.com/pierrec/lz4 v2.5.2+incompatible go: downloading github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: downloading github.com/fvbommel/sortorder v1.0.1 go: downloading github.com/go-errors/errors v1.0.1 go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 go: downloading github.com/xlab/treeprint v0.0.0-20181112141820-a009c3971eca go: downloading github.com/go-openapi/jsonreference v0.19.5 go: downloading github.com/go-openapi/swag v0.19.14 go: downloading github.com/armon/go-metrics v0.3.3 go: downloading github.com/hashicorp/go-msgpack v0.5.3 go: downloading github.com/miekg/dns v1.1.35 go: downloading github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 go: downloading github.com/creack/pty v1.1.11 go: downloading gotest.tools/v3 v3.0.3 go: downloading gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 go: downloading github.com/jmespath/go-jmespath/internal/testify v1.5.1 go: downloading github.com/jtolds/gls v4.20.0+incompatible go: downloading github.com/smartystreets/assertions v1.1.0 go: downloading gotest.tools v2.2.0+incompatible go: downloading github.com/docker/docker v20.10.10+incompatible go: downloading github.com/google/certificate-transparency-go v1.1.1 go: downloading github.com/google/go-tspi v0.2.1-0.20190423175329-115dea689aad go: downloading github.com/moby/spdystream v0.2.0 go: downloading github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153 go: downloading github.com/go-openapi/jsonpointer v0.19.5 go: downloading github.com/mailru/easyjson v0.7.6 go: downloading github.com/PuerkitoBio/purell v1.1.1 go: downloading github.com/hashicorp/go-immutable-radix v1.1.0 go: downloading github.com/pascaldekloe/goe v0.1.0 go: downloading github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 go: downloading github.com/gopherjs/gopherjs v0.0.0-20200217142428-fce0ec30dd00 go: downloading github.com/bshuster-repo/logrus-logstash-hook v1.0.0 go: downloading github.com/bugsnag/bugsnag-go v0.0.0-20141110184014-b1d153021fcd go: downloading github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d go: downloading github.com/docker/go-metrics v0.0.1 go: downloading github.com/docker/libtrust v0.0.0-20150114040149-fa567046d9b1 go: downloading github.com/gorilla/handlers v1.5.1 go: downloading github.com/yvasiyarov/gorelic v0.0.0-20141212073537-a9bba5b9ab50 go: downloading rsc.io/letsencrypt v0.0.3 go: downloading github.com/docker/docker-credential-helpers v0.6.4 go: downloading go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 go: downloading github.com/moby/locker v1.0.1 go: downloading github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 go: downloading github.com/hashicorp/golang-lru v0.5.4 go: downloading github.com/hashicorp/go-uuid v1.0.2 go: downloading github.com/josharian/intern v1.0.0 go: downloading github.com/gorilla/mux v1.8.0 go: downloading github.com/garyburd/redigo v0.0.0-20150301180006-535138d7bcd7 go: downloading github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0 go: downloading github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c go: downloading github.com/gomodule/redigo v1.8.2 go: downloading github.com/yvasiyarov/go-metrics v0.0.0-20140926110328-57bccd1ccd43 go: downloading github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f go: downloading github.com/felixge/httpsnoop v1.0.1 go: downloading github.com/bugsnag/osext v0.0.0-20130617224835-0dd3f918b21b go: downloading github.com/Microsoft/hcsshim v0.8.22 go: downloading github.com/moby/sys/mountinfo v0.4.1 go: downloading github.com/Microsoft/go-winio v0.5.1 go: downloading golang.org/x/mod v0.5.1 go: downloading github.com/containerd/cgroups v1.0.1 go: downloading github.com/docker/go-units v0.4.0 go: downloading github.com/docker/go-connections v0.4.0 go: downloading github.com/morikuni/aec v1.0.0 go: downloading github.com/frankban/quicktest v1.11.3 go: downloading github.com/Azure/go-autorest/autorest/validation v0.3.1 go: downloading github.com/Masterminds/sprig v2.22.0+incompatible go: downloading github.com/cpuguy83/go-md2man v1.0.10 go: downloading github.com/Masterminds/semver v1.5.0 go: downloading github.com/cespare/xxhash v1.1.0 go: downloading github.com/apparentlymart/go-textseg v1.0.0 /bin/sh: tools/get_version.sh: not found go build -ldflags=all="-X k8s.io/kops.Version= -X k8s.io/kops.GitVersion=v20220328-3613-g6a71ba7593 " -o /home/buildozer/aports/testing/kops/src/kops-1.23.0/.build/local/kops k8s.io/kops/cmd/kops/ go test -v ./... ? k8s.io/kops [no test files] ? k8s.io/kops/channels/cmd/channels [no test files] ? k8s.io/kops/channels/pkg/api [no test files] === RUN Test_Filtering --- PASS: Test_Filtering (0.00s) === RUN Test_Replacement --- PASS: Test_Replacement (0.00s) === RUN Test_GetRequiredUpdates --- PASS: Test_GetRequiredUpdates (0.00s) === RUN Test_NeedsRollingUpdate I0512 07:42:00.847570 5517 addon.go:218] addon test wants to update all nodes I0512 07:42:00.849520 5517 addon.go:218] addon test wants to update worker nodes I0512 07:42:00.849902 5517 addon.go:218] addon test wants to update control-plane nodes --- PASS: Test_NeedsRollingUpdate (0.00s) === RUN Test_InstallPKI I0512 07:42:00.850400 5517 addon.go:251] installing PKI for "test" I0512 07:42:01.485588 5517 addon.go:251] installing PKI for "test" --- PASS: Test_InstallPKI (1.12s) === RUN Test_IsPKIInstalled --- PASS: Test_IsPKIInstalled (0.00s) PASS ok k8s.io/kops/channels/pkg/channels 1.231s ? k8s.io/kops/channels/pkg/cmd [no test files] ? k8s.io/kops/cloudmock/aws/mockautoscaling [no test files] ? k8s.io/kops/cloudmock/aws/mockec2 [no test files] ? k8s.io/kops/cloudmock/aws/mockelb [no test files] ? k8s.io/kops/cloudmock/aws/mockelbv2 [no test files] ? k8s.io/kops/cloudmock/aws/mockeventbridge [no test files] ? k8s.io/kops/cloudmock/aws/mockiam [no test files] ? k8s.io/kops/cloudmock/aws/mockroute53 [no test files] ? k8s.io/kops/cloudmock/aws/mocksqs [no test files] ? k8s.io/kops/cloudmock/gce [no test files] ? k8s.io/kops/cloudmock/gce/mockcompute [no test files] ? k8s.io/kops/cloudmock/gce/mockdns [no test files] ? k8s.io/kops/cloudmock/openstack [no test files] ? k8s.io/kops/cloudmock/openstack/mockblockstorage [no test files] ? k8s.io/kops/cloudmock/openstack/mockcompute [no test files] ? k8s.io/kops/cloudmock/openstack/mockdns [no test files] ? k8s.io/kops/cloudmock/openstack/mockimage [no test files] ? k8s.io/kops/cloudmock/openstack/mockloadbalancer [no test files] ? k8s.io/kops/cloudmock/openstack/mocknetworking [no test files] === RUN TestCreateClusterMinimal I0512 07:42:19.334410 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:19.334515 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:19.334561 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:19.334615 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:19.334657 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:19.334674 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:19.334705 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:19.334735 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:19.334754 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:19.334805 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:19.334847 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:19.392254 8834 new_cluster.go:1102] Cloud Provider ID = aws I0512 07:42:19.392367 8834 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0512 07:42:19.392739 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:19.413398 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:19.413460 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:19.413510 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:19.413573 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:19.413603 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:19.413651 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:19.413682 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:19.413720 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:19.413755 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:19.413825 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:19.413868 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:19.477615 8834 new_cluster.go:1102] Cloud Provider ID = aws I0512 07:42:19.477683 8834 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0512 07:42:19.477932 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:19.483274 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:19.483302 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:19.483325 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:19.483350 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:19.483372 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:19.483390 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:19.483408 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:19.483423 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:19.483438 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:19.483472 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:19.483494 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:19.509011 8834 new_cluster.go:1102] Cloud Provider ID = aws I0512 07:42:19.509060 8834 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0512 07:42:19.509233 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:19.515859 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:19.515925 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:19.516219 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:19.516255 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:19.516290 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:19.516308 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:19.516327 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:19.516357 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:19.516374 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:19.516419 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:19.516443 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:19.559418 8834 new_cluster.go:1102] Cloud Provider ID = aws I0512 07:42:19.559500 8834 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0512 07:42:19.559779 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:19.565481 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:19.565512 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:19.565543 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:19.565573 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:19.565601 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:19.565618 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:19.565640 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:19.565666 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:19.565685 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:19.565724 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:19.565750 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:19.598764 8834 new_cluster.go:1102] Cloud Provider ID = aws I0512 07:42:19.598847 8834 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0512 07:42:19.599134 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:19.604983 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:19.605016 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:19.605043 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:19.605074 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:19.605100 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:19.605118 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:19.605136 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:19.605157 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:19.605175 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:19.605221 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:19.605244 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:19.643799 8834 new_cluster.go:1102] Cloud Provider ID = aws I0512 07:42:19.643885 8834 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0512 07:42:19.644167 8834 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterMinimal (0.32s) === RUN TestCreateClusterOverride I0512 07:42:19.653883 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:19.653929 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:19.653967 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:19.654007 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:19.654040 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:19.654063 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:19.654090 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:19.654119 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:19.654141 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:19.654207 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:19.654238 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:19.707109 8834 new_cluster.go:1102] Cloud Provider ID = aws I0512 07:42:19.707588 8834 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0512 07:42:19.707939 8834 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterOverride (0.06s) === RUN TestCreateClusterComplex I0512 07:42:19.714938 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:19.714968 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:19.714994 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:19.715023 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:19.715049 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:19.715069 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:19.715089 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:19.715110 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:19.715127 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:19.715176 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:19.715203 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:19.746213 8834 new_cluster.go:1102] Cloud Provider ID = aws I0512 07:42:19.746297 8834 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0512 07:42:19.746571 8834 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterComplex (0.04s) === RUN TestCreateClusterHA I0512 07:42:19.752405 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:19.752436 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:19.752462 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:19.752491 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:19.752516 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:19.752537 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:19.752556 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:19.752578 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:19.752594 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:19.752652 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:19.752677 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:19.810730 8834 new_cluster.go:1102] Cloud Provider ID = aws I0512 07:42:19.810814 8834 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0512 07:42:19.810827 8834 subnets.go:182] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0512 07:42:19.810837 8834 subnets.go:182] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0512 07:42:19.811154 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:19.822499 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:19.822549 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:19.822579 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:19.822608 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:19.822635 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:19.822652 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:19.822670 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:19.822690 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:19.822712 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:19.822756 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:19.822779 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:19.872500 8834 new_cluster.go:1102] Cloud Provider ID = aws I0512 07:42:19.872584 8834 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0512 07:42:19.872596 8834 subnets.go:182] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0512 07:42:19.872606 8834 subnets.go:182] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0512 07:42:19.872929 8834 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHA (0.13s) === RUN TestCreateClusterHAGCE I0512 07:42:19.882336 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:19.882371 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:19.882399 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:19.882427 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:19.882452 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:19.882470 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:19.882488 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:19.882511 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:19.882528 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:19.882582 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:19.882607 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0512 07:42:19.919643 8834 new_cluster.go:372] VMs will be configured to use the GCE default compute Service Account! This is an anti-pattern W0512 07:42:19.919679 8834 new_cluster.go:373] Use a pre-created Service Account with the flag: --gce-service-account=account@projectname.iam.gserviceaccount.com I0512 07:42:19.919754 8834 new_cluster.go:1102] Cloud Provider ID = gce I0512 07:42:19.919895 8834 network.go:198] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterHAGCE (0.05s) === RUN TestCreateClusterGCE I0512 07:42:19.936146 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:19.936200 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:19.936250 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:19.936302 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:19.936345 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:19.936375 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:19.936409 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:19.936452 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:19.936482 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:19.936561 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:19.936604 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:19.992822 8834 new_cluster.go:369] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0512 07:42:19.992873 8834 new_cluster.go:1102] Cloud Provider ID = gce I0512 07:42:19.992902 8834 network.go:198] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterGCE (0.06s) === RUN TestCreateClusterHASharedZone I0512 07:42:19.998831 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:19.998858 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:19.998880 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:19.998908 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:19.998930 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:19.998947 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:19.998966 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:19.998984 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:19.998999 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:19.999039 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:19.999063 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0512 07:42:20.080598 8834 new_cluster.go:750] Running with masters in the same AZs; redundancy will be reduced I0512 07:42:20.080709 8834 new_cluster.go:1102] Cloud Provider ID = aws I0512 07:42:20.080841 8834 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0512 07:42:20.081881 8834 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZone (0.10s) === RUN TestCreateClusterHASharedZones I0512 07:42:20.094589 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:20.094635 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:20.094683 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:20.094732 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:20.094770 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:20.094800 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:20.094829 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:20.094858 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:20.094882 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:20.094955 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:20.094994 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0512 07:42:20.153092 8834 new_cluster.go:750] Running with masters in the same AZs; redundancy will be reduced I0512 07:42:20.153141 8834 new_cluster.go:1102] Cloud Provider ID = aws I0512 07:42:20.153200 8834 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0512 07:42:20.153216 8834 subnets.go:182] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0512 07:42:20.153459 8834 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZones (0.07s) === RUN TestCreateClusterPrivate I0512 07:42:20.163760 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:20.163787 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:20.163810 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:20.163833 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:20.163857 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:20.163873 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:20.163890 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:20.163908 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:20.163922 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:20.163974 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:20.163997 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:20.240426 8834 new_cluster.go:1102] Cloud Provider ID = aws I0512 07:42:20.240521 8834 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0512 07:42:20.240536 8834 subnets.go:196] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0512 07:42:20.240841 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:20.240884 8834 utils.go:203] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterPrivate (0.08s) === RUN TestCreateClusterPrivateGCE I0512 07:42:20.248068 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:20.248098 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:20.248127 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:20.248156 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:20.248180 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:20.248200 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:20.248219 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:20.248240 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:20.248257 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:20.248307 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:20.248334 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:20.298767 8834 new_cluster.go:369] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0512 07:42:20.298840 8834 new_cluster.go:1102] Cloud Provider ID = gce I0512 07:42:20.298891 8834 network.go:198] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterPrivateGCE (0.06s) === RUN TestCreateClusterWithNGWSpecified I0512 07:42:20.308090 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:20.308134 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:20.308169 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:20.308198 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:20.308231 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:20.308251 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:20.308269 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:20.308300 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:20.308317 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:20.308377 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:20.308402 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:20.374446 8834 new_cluster.go:1102] Cloud Provider ID = aws I0512 07:42:20.374555 8834 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0512 07:42:20.374572 8834 subnets.go:196] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0512 07:42:20.374873 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:20.374906 8834 utils.go:203] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithNGWSpecified (0.07s) === RUN TestCreateClusterWithINGWSpecified I0512 07:42:20.381717 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:20.381750 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:20.381778 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:20.381806 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:20.381834 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:20.381851 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:20.381871 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:20.381901 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:20.381918 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:20.381969 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:20.381994 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:20.494872 8834 new_cluster.go:1102] Cloud Provider ID = aws I0512 07:42:20.494968 8834 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0512 07:42:20.494987 8834 subnets.go:196] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0512 07:42:20.495268 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:20.495302 8834 utils.go:203] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithINGWSpecified (0.12s) === RUN TestCreateClusterSharedVPC I0512 07:42:20.502480 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:20.502510 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:20.502538 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:20.502567 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:20.502593 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:20.502611 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:20.502631 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:20.502651 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:20.502667 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:20.502714 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:20.502737 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:20.560081 8834 new_cluster.go:1102] Cloud Provider ID = aws I0512 07:42:20.560122 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0512 07:42:20.560165 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0512 07:42:20.560196 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0512 07:42:20.560212 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0512 07:42:20.560246 8834 subnets.go:182] Assigned CIDR 10.2.0.0/15 to subnet us-test-1a I0512 07:42:20.560425 8834 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedVPC (0.07s) === RUN TestCreateClusterSharedSubnets I0512 07:42:20.570094 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:20.570143 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:20.570189 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:20.570235 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:20.570273 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:20.570302 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:20.570330 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:20.570357 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:20.570382 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:20.570451 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:20.570490 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:20.631917 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0512 07:42:20.631952 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0512 07:42:20.632015 8834 new_cluster.go:1102] Cloud Provider ID = aws I0512 07:42:20.632054 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0512 07:42:20.632078 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0512 07:42:20.632106 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0512 07:42:20.632125 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0512 07:42:20.632359 8834 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnets (0.07s) === RUN TestCreateClusterSharedSubnetsVpcLookup I0512 07:42:20.638404 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:20.638433 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:20.638457 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:20.638482 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:20.638505 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:20.638523 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:20.638542 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:20.638559 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:20.638572 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:20.638614 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:20.638637 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:20.674343 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:42:20.674374 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0512 07:42:20.674391 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0512 07:42:20.674458 8834 new_cluster.go:1102] Cloud Provider ID = aws I0512 07:42:20.674502 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0512 07:42:20.674522 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0512 07:42:20.674547 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0512 07:42:20.674562 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0512 07:42:20.674755 8834 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnetsVpcLookup (0.04s) === RUN TestCreateClusterPrivateSharedSubnets I0512 07:42:20.679817 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:20.679843 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:20.679864 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:20.679887 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:20.679907 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:20.679923 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:20.679942 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:20.679958 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:20.679983 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:20.680022 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:20.680044 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:20.760048 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0512 07:42:20.760084 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0512 07:42:20.760152 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0512 07:42:20.760174 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0512 07:42:20.760215 8834 new_cluster.go:1102] Cloud Provider ID = aws I0512 07:42:20.760263 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0512 07:42:20.760281 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0512 07:42:20.760316 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0512 07:42:20.760334 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0512 07:42:20.760571 8834 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterPrivateSharedSubnets (0.09s) === RUN TestCreateClusterIPv6 I0512 07:42:20.766237 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:20.766264 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:20.766288 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:20.766313 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:20.766335 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:20.766352 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:20.766368 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:20.766384 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:20.766398 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:20.766438 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:20.766461 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0512 07:42:20.787501 8834 new_cluster.go:1102] Cloud Provider ID = aws I0512 07:42:20.787553 8834 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0512 07:42:20.787830 8834 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterIPv6 (0.03s) === RUN TestParseCloudLabels --- PASS: TestParseCloudLabels (0.00s) === RUN TestConfirmation --- PASS: TestConfirmation (0.00s) === RUN TestMinimal I0512 07:42:20.793438 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:20.793463 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:20.793485 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:20.794040 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:20.794252 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:20.794301 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:20.794342 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:20.794388 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:20.794429 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:20.818550 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:20.821119 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:20.821254 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0512 07:42:23.595879 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:26.554058 8834 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0512 07:42:26.554928 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:26.555796 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:26.556023 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:26.556120 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:26.556150 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:26.556267 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0512 07:42:26.558785 8834 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0512 07:42:26.562587 8834 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0512 07:42:26.564350 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:26.564581 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:26.566844 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:26.567722 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:26.567825 8834 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0512 07:42:26.568159 8834 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0512 07:42:26.609812 8834 target.go:112] Terraform output is in /tmp/test2004962075/out I0512 07:42:26.614113 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:26.614188 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:26.614248 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:26.614355 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:26.614429 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:26.614517 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:26.614592 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:26.614645 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:26.614688 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:26.637631 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:26.640387 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:26.640477 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0512 07:42:26.679607 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0512 07:42:26.746995 8834 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0512 07:42:26.747528 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:26.747610 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:26.747750 8834 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0512 07:42:26.747787 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0512 07:42:26.747873 8834 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0512 07:42:26.747792 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:26.747968 8834 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0512 07:42:26.747910 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:26.748058 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0512 07:42:26.753213 8834 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0512 07:42:26.756245 8834 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0512 07:42:26.756635 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:26.756870 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:26.758022 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:26.759030 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:26.759192 8834 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0512 07:42:26.759491 8834 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0512 07:42:26.764397 8834 target.go:168] Cloudformation output is in /tmp/test451309580/out --- PASS: TestMinimal (5.98s) === RUN TestMinimal_v1_23 I0512 07:42:26.777728 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:26.777888 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:26.778023 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:26.778107 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:26.778205 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:26.778241 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:26.778272 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:26.778307 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:26.778339 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:26.803767 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* I0512 07:42:26.806834 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:26.806929 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:28.276792 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:28.445587 8834 executor.go:111] Tasks: 0 done / 91 total; 46 can run I0512 07:42:28.446423 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0512 07:42:28.446480 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:28.446463 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:28.446565 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0512 07:42:28.446583 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:28.446965 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:28.448498 8834 executor.go:111] Tasks: 46 done / 91 total; 19 can run I0512 07:42:28.453968 8834 executor.go:111] Tasks: 65 done / 91 total; 24 can run I0512 07:42:28.454413 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:28.454609 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:28.455920 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:28.457511 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:28.457637 8834 executor.go:111] Tasks: 89 done / 91 total; 2 can run I0512 07:42:28.458947 8834 executor.go:111] Tasks: 91 done / 91 total; 0 can run I0512 07:42:28.516556 8834 target.go:112] Terraform output is in /tmp/test588262297/out I0512 07:42:28.521198 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:28.521257 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:28.521305 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:28.521358 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:28.521401 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:28.521434 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:28.521468 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:28.521504 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:28.521534 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:28.544324 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:28.547418 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:28.547512 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0512 07:42:28.585800 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0512 07:42:28.689315 8834 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0512 07:42:28.689840 8834 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0512 07:42:28.689904 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:28.689874 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:28.689975 8834 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0512 07:42:28.689976 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:28.690137 8834 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0512 07:42:28.690164 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:28.690279 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:28.690302 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0512 07:42:28.695740 8834 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0512 07:42:28.701351 8834 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0512 07:42:28.701721 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:28.702017 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:28.702808 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:28.704820 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:28.704961 8834 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0512 07:42:28.705262 8834 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0512 07:42:28.708135 8834 target.go:168] Cloudformation output is in /tmp/test3483911859/out --- PASS: TestMinimal_v1_23 (1.94s) === RUN TestNvidia I0512 07:42:28.718180 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:28.718230 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:28.718272 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:28.718320 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:28.718400 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:28.718429 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:28.718463 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:28.718497 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:28.718528 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:28.745574 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:28.748313 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:28.748409 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:28.786110 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:28.898289 8834 executor.go:111] Tasks: 0 done / 87 total; 45 can run W0512 07:42:28.898735 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:28.899167 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:28.899285 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:28.899273 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:28.899337 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:28.899440 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0512 07:42:28.901503 8834 executor.go:111] Tasks: 45 done / 87 total; 19 can run I0512 07:42:28.910574 8834 executor.go:111] Tasks: 64 done / 87 total; 21 can run I0512 07:42:28.910951 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:28.911174 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:28.912321 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:28.913414 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:28.913533 8834 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0512 07:42:28.913974 8834 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0512 07:42:28.968640 8834 target.go:112] Terraform output is in /tmp/test3651839732/out I0512 07:42:28.972873 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:28.972932 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:28.972980 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:28.973035 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:28.973076 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:28.973105 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:28.973136 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:28.973173 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:28.973203 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:28.997441 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:29.000338 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:29.000438 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:29.037534 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0512 07:42:29.148960 8834 executor.go:111] Tasks: 0 done / 87 total; 45 can run W0512 07:42:29.149368 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0512 07:42:29.149660 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:29.149928 8834 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0512 07:42:29.149969 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:29.149970 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:29.150059 8834 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0512 07:42:29.150112 8834 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0512 07:42:29.150131 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:29.150217 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0512 07:42:29.152206 8834 executor.go:111] Tasks: 45 done / 87 total; 19 can run I0512 07:42:29.158590 8834 executor.go:111] Tasks: 64 done / 87 total; 21 can run I0512 07:42:29.159048 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:29.159247 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:29.160390 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:29.161482 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:29.161624 8834 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0512 07:42:29.161870 8834 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0512 07:42:29.164884 8834 target.go:168] Cloudformation output is in /tmp/test2206800193/out --- PASS: TestNvidia (0.46s) === RUN TestMinimalGossip I0512 07:42:29.177506 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:29.177572 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:29.177621 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:29.177672 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:29.177716 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:29.177745 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:29.177775 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:29.177835 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:29.177885 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:29.204399 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:29.204511 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:29.241977 8834 apply_cluster.go:483] Gossip DNS: skipping DNS validation I0512 07:42:29.348865 8834 executor.go:111] Tasks: 0 done / 85 total; 43 can run W0512 07:42:29.349117 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:29.349392 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:29.349761 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:29.355923 8834 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0512 07:42:29.363566 8834 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0512 07:42:29.363850 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:29.364095 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:29.365253 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:29.366270 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:29.366385 8834 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0512 07:42:29.366699 8834 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0512 07:42:29.422320 8834 target.go:112] Terraform output is in /tmp/test2739856283/out --- PASS: TestMinimalGossip (0.25s) === RUN TestMinimalGossipIRSA I0512 07:42:29.426569 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:29.426628 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:29.426681 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:29.426737 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:29.426782 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:29.426812 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:29.426846 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:29.426882 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:29.426912 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:29.454329 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:29.454429 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:29.492306 8834 apply_cluster.go:483] Gossip DNS: skipping DNS validation I0512 07:42:29.606012 8834 executor.go:111] Tasks: 0 done / 88 total; 45 can run W0512 07:42:29.606610 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:29.606782 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:29.609825 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:29.612983 8834 executor.go:111] Tasks: 45 done / 88 total; 20 can run I0512 07:42:29.618481 8834 executor.go:111] Tasks: 65 done / 88 total; 21 can run I0512 07:42:29.618773 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:29.619081 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:29.620137 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:29.621119 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:29.621276 8834 executor.go:111] Tasks: 86 done / 88 total; 2 can run I0512 07:42:29.621634 8834 executor.go:111] Tasks: 88 done / 88 total; 0 can run I0512 07:42:29.678257 8834 target.go:112] Terraform output is in /tmp/test3396824745/out --- PASS: TestMinimalGossipIRSA (0.26s) === RUN TestMinimalGCE I0512 07:42:29.682642 8834 featureflag.go:162] FeatureFlag "AlphaAllowGCE"=true I0512 07:42:29.684523 8834 network.go:198] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* W0512 07:42:30.552978 8834 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0512 07:42:30.553069 8834 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0512 07:42:30.553201 8834 autoscalinggroup.go:118] enabling storage-rw for etcd backups W0512 07:42:30.553242 8834 autoscalinggroup.go:163] VMs will be configured to use the GCE default compute Service Account! This is an anti-pattern W0512 07:42:30.553265 8834 autoscalinggroup.go:164] Use a pre-created Service Account with the flag: --gce-service-account=account@projectname.iam.gserviceaccount.com I0512 07:42:30.553284 8834 autoscalinggroup.go:168] gsa: default I0512 07:42:30.553359 8834 autoscalinggroup.go:160] VMs using Service Account: default I0512 07:42:30.553383 8834 autoscalinggroup.go:168] gsa: default I0512 07:42:30.554996 8834 executor.go:111] Tasks: 0 done / 61 total; 38 can run I0512 07:42:30.561550 8834 executor.go:111] Tasks: 38 done / 61 total; 17 can run I0512 07:42:30.566999 8834 executor.go:111] Tasks: 55 done / 61 total; 4 can run I0512 07:42:30.568526 8834 instancetemplate.go:555] adding csa: default I0512 07:42:30.570135 8834 instancetemplate.go:555] adding csa: default I0512 07:42:30.570337 8834 executor.go:111] Tasks: 59 done / 61 total; 2 can run I0512 07:42:30.570685 8834 executor.go:111] Tasks: 61 done / 61 total; 0 can run I0512 07:42:30.601183 8834 target.go:112] Terraform output is in /tmp/test1778865119/out --- PASS: TestMinimalGCE (0.92s) === RUN TestMinimalGCEPrivate I0512 07:42:30.605558 8834 featureflag.go:162] FeatureFlag "AlphaAllowGCE"=true I0512 07:42:30.607372 8834 network.go:198] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* W0512 07:42:31.417195 8834 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0512 07:42:31.417289 8834 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0512 07:42:31.417438 8834 autoscalinggroup.go:118] enabling storage-rw for etcd backups W0512 07:42:31.417482 8834 autoscalinggroup.go:163] VMs will be configured to use the GCE default compute Service Account! This is an anti-pattern W0512 07:42:31.417504 8834 autoscalinggroup.go:164] Use a pre-created Service Account with the flag: --gce-service-account=account@projectname.iam.gserviceaccount.com I0512 07:42:31.417525 8834 autoscalinggroup.go:168] gsa: default I0512 07:42:31.417601 8834 autoscalinggroup.go:160] VMs using Service Account: default I0512 07:42:31.417625 8834 autoscalinggroup.go:168] gsa: default I0512 07:42:31.419248 8834 executor.go:111] Tasks: 0 done / 63 total; 37 can run I0512 07:42:31.422915 8834 executor.go:111] Tasks: 37 done / 63 total; 18 can run I0512 07:42:31.431702 8834 executor.go:111] Tasks: 55 done / 63 total; 4 can run I0512 07:42:31.434319 8834 instancetemplate.go:555] adding csa: default I0512 07:42:31.434456 8834 executor.go:111] Tasks: 59 done / 63 total; 3 can run I0512 07:42:31.435765 8834 instancetemplate.go:555] adding csa: default I0512 07:42:31.435874 8834 executor.go:111] Tasks: 62 done / 63 total; 1 can run I0512 07:42:31.436008 8834 executor.go:111] Tasks: 63 done / 63 total; 0 can run I0512 07:42:31.468911 8834 target.go:112] Terraform output is in /tmp/test2474696479/out --- PASS: TestMinimalGCEPrivate (0.87s) === RUN TestHA I0512 07:42:31.472993 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:31.473050 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:31.473101 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:31.473158 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:31.473203 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:31.473232 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:31.473263 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:31.473298 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:31.473328 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:31.505006 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:31.508083 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:31.508174 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:31.508231 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:31.508290 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:31.545103 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:31.650263 8834 executor.go:111] Tasks: 0 done / 104 total; 50 can run I0512 07:42:31.650614 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.ha.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:31.650856 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:31.650919 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:31.650954 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:31.651103 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0512 07:42:31.654009 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:31.654044 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:31.654076 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:31.658252 8834 executor.go:111] Tasks: 50 done / 104 total; 23 can run I0512 07:42:31.663611 8834 executor.go:111] Tasks: 73 done / 104 total; 27 can run I0512 07:42:31.663880 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:31.664146 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:31.664294 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:31.665493 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:31.669217 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:31.670492 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:31.670775 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:31.671508 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:31.671641 8834 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0512 07:42:31.672168 8834 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0512 07:42:31.746847 8834 target.go:112] Terraform output is in /tmp/test2093815801/out --- PASS: TestHA (0.28s) === RUN TestHighAvailabilityGCE I0512 07:42:31.751622 8834 featureflag.go:162] FeatureFlag "AlphaAllowGCE"=true I0512 07:42:31.753677 8834 network.go:198] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* W0512 07:42:31.935893 8834 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0512 07:42:31.935982 8834 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0512 07:42:31.936121 8834 autoscalinggroup.go:118] enabling storage-rw for etcd backups W0512 07:42:31.936161 8834 autoscalinggroup.go:163] VMs will be configured to use the GCE default compute Service Account! This is an anti-pattern W0512 07:42:31.936184 8834 autoscalinggroup.go:164] Use a pre-created Service Account with the flag: --gce-service-account=account@projectname.iam.gserviceaccount.com I0512 07:42:31.936204 8834 autoscalinggroup.go:168] gsa: default W0512 07:42:31.936262 8834 autoscalinggroup.go:118] enabling storage-rw for etcd backups I0512 07:42:31.936294 8834 autoscalinggroup.go:160] VMs using Service Account: default I0512 07:42:31.936314 8834 autoscalinggroup.go:168] gsa: default W0512 07:42:31.936364 8834 autoscalinggroup.go:118] enabling storage-rw for etcd backups I0512 07:42:31.936400 8834 autoscalinggroup.go:160] VMs using Service Account: default I0512 07:42:31.936422 8834 autoscalinggroup.go:168] gsa: default I0512 07:42:31.936483 8834 autoscalinggroup.go:160] VMs using Service Account: default I0512 07:42:31.936505 8834 autoscalinggroup.go:168] gsa: default I0512 07:42:31.938522 8834 executor.go:111] Tasks: 0 done / 76 total; 41 can run I0512 07:42:31.945386 8834 executor.go:111] Tasks: 41 done / 76 total; 20 can run I0512 07:42:31.951677 8834 executor.go:111] Tasks: 61 done / 76 total; 7 can run I0512 07:42:31.954299 8834 instancetemplate.go:555] adding csa: default I0512 07:42:31.954390 8834 instancetemplate.go:555] adding csa: default I0512 07:42:31.954423 8834 instancetemplate.go:555] adding csa: default I0512 07:42:31.954649 8834 executor.go:111] Tasks: 68 done / 76 total; 5 can run I0512 07:42:31.959560 8834 instancetemplate.go:555] adding csa: default I0512 07:42:31.959691 8834 executor.go:111] Tasks: 73 done / 76 total; 3 can run I0512 07:42:31.959987 8834 executor.go:111] Tasks: 76 done / 76 total; 0 can run I0512 07:42:32.001043 8834 target.go:112] Terraform output is in /tmp/test859324172/out --- PASS: TestHighAvailabilityGCE (0.25s) === RUN TestComplex I0512 07:42:32.006136 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:32.006196 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:32.006252 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:32.006307 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:32.006398 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:32.006428 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:32.006460 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:32.006496 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:32.006525 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:32.040353 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:32.044253 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:32.044382 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:32.083755 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:32.194681 8834 executor.go:111] Tasks: 0 done / 116 total; 48 can run W0512 07:42:32.195613 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:32.195913 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:32.195937 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:32.195989 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:32.196147 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0512 07:42:32.198125 8834 executor.go:111] Tasks: 48 done / 116 total; 27 can run I0512 07:42:32.207161 8834 executor.go:111] Tasks: 75 done / 116 total; 38 can run I0512 07:42:32.207478 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:32.207775 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:32.208737 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:32.210154 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:32.210341 8834 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0512 07:42:32.210966 8834 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0512 07:42:32.284977 8834 target.go:112] Terraform output is in /tmp/test667942891/out I0512 07:42:32.289071 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:32.289132 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:32.289182 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:32.289233 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:32.289275 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:32.289304 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:32.289335 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:32.289373 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:32.289402 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:32.321090 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:32.324655 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:32.324743 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:32.362599 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0512 07:42:32.466837 8834 executor.go:111] Tasks: 0 done / 116 total; 48 can run W0512 07:42:32.468360 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0512 07:42:32.468628 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:32.468726 8834 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0512 07:42:32.468771 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:32.468951 8834 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0512 07:42:32.473479 8834 executor.go:111] Tasks: 48 done / 116 total; 27 can run I0512 07:42:32.479389 8834 executor.go:111] Tasks: 75 done / 116 total; 38 can run I0512 07:42:32.480186 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:32.480461 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:32.481472 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:32.483118 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:32.483262 8834 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0512 07:42:32.483702 8834 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0512 07:42:32.488194 8834 target.go:168] Cloudformation output is in /tmp/test2775072398/out I0512 07:42:32.506175 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:32.506231 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:32.506277 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:32.506326 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:32.506367 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:32.506397 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:32.506427 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:32.506463 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:32.506491 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:32.537702 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:32.541248 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:32.541334 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:32.580862 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:32.678741 8834 executor.go:111] Tasks: 0 done / 116 total; 48 can run I0512 07:42:32.679028 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:32.679061 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:32.679156 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0512 07:42:32.679666 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:32.679700 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:32.681445 8834 executor.go:111] Tasks: 48 done / 116 total; 27 can run I0512 07:42:32.689725 8834 executor.go:111] Tasks: 75 done / 116 total; 38 can run I0512 07:42:32.690180 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:32.690502 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:32.691608 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:32.692819 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:32.692938 8834 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0512 07:42:32.693469 8834 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0512 07:42:32.767668 8834 target.go:112] Terraform output is in /tmp/test1932423396/out --- PASS: TestComplex (0.77s) === RUN TestCompress I0512 07:42:32.771924 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:32.771986 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:32.772040 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:32.772097 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:32.772139 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:32.772168 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:32.772198 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:32.772232 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:32.772261 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:32.796473 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:32.799331 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:32.799427 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:32.836865 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:32.938639 8834 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0512 07:42:32.939420 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0512 07:42:32.939580 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:32.941399 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:32.939477 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:32.942069 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0512 07:42:32.946393 8834 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0512 07:42:32.953394 8834 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0512 07:42:32.953926 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:32.954221 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:32.959188 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:32.962832 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:32.962973 8834 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0512 07:42:32.963374 8834 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0512 07:42:33.009395 8834 target.go:112] Terraform output is in /tmp/test2593026999/out --- PASS: TestCompress (0.24s) === RUN TestExternalPolicies I0512 07:42:33.014882 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:33.015031 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:33.015300 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:33.015389 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:33.015445 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:33.015490 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:33.015530 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:33.015581 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:33.015650 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:33.045836 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:33.049507 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:33.049612 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:33.086258 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:33.189975 8834 executor.go:111] Tasks: 0 done / 99 total; 46 can run I0512 07:42:33.190388 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externalpolicies.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:33.190437 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:33.190560 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:33.190835 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:33.190860 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:33.190997 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0512 07:42:33.197576 8834 executor.go:111] Tasks: 46 done / 99 total; 20 can run I0512 07:42:33.202978 8834 executor.go:111] Tasks: 66 done / 99 total; 30 can run I0512 07:42:33.203481 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:33.203735 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:33.204691 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:33.206323 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:33.206447 8834 executor.go:111] Tasks: 96 done / 99 total; 3 can run I0512 07:42:33.206844 8834 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0512 07:42:33.275348 8834 target.go:112] Terraform output is in /tmp/test3219995158/out --- PASS: TestExternalPolicies (0.27s) === RUN TestMinimalIPv6 I0512 07:42:33.279891 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:33.279959 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:33.280011 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:33.280066 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:33.280153 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:33.280182 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:33.280213 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:33.280248 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:33.280278 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:33.305811 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:33.308803 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:33.308893 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:33.345875 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:33.523377 8834 executor.go:111] Tasks: 0 done / 105 total; 46 can run I0512 07:42:33.524309 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:33.524348 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:33.524396 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:33.524453 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:33.524457 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0512 07:42:33.524535 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:33.526230 8834 executor.go:111] Tasks: 46 done / 105 total; 21 can run I0512 07:42:33.531750 8834 executor.go:111] Tasks: 67 done / 105 total; 30 can run I0512 07:42:33.532308 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:33.532519 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:33.537785 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:33.539162 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:33.539308 8834 executor.go:111] Tasks: 97 done / 105 total; 5 can run I0512 07:42:33.540035 8834 executor.go:111] Tasks: 102 done / 105 total; 3 can run I0512 07:42:33.540429 8834 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0512 07:42:33.603614 8834 target.go:112] Terraform output is in /tmp/test2082097792/out I0512 07:42:33.608180 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:33.608241 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:33.608290 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:33.608347 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:33.608391 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:33.608421 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:33.608452 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:33.608489 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:33.608516 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:33.633699 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:33.636972 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:33.637069 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:33.674169 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0512 07:42:33.848758 8834 executor.go:111] Tasks: 0 done / 105 total; 46 can run W0512 07:42:33.849301 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:33.849295 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:33.849371 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0512 07:42:33.849478 8834 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0512 07:42:33.849502 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:33.849613 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "minimal-ipv6.example.com" }] }] } I0512 07:42:33.849820 8834 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0512 07:42:33.849850 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:33.849989 8834 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0512 07:42:33.853579 8834 executor.go:111] Tasks: 46 done / 105 total; 21 can run I0512 07:42:33.861470 8834 executor.go:111] Tasks: 67 done / 105 total; 30 can run I0512 07:42:33.861840 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:33.862122 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:33.863062 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:33.864676 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:33.864833 8834 executor.go:111] Tasks: 97 done / 105 total; 5 can run I0512 07:42:33.865282 8834 executor.go:111] Tasks: 102 done / 105 total; 3 can run I0512 07:42:33.865593 8834 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0512 07:42:33.869698 8834 target.go:168] Cloudformation output is in /tmp/test1984730554/out --- PASS: TestMinimalIPv6 (0.60s) === RUN TestMinimalIPv6Private I0512 07:42:33.882493 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:33.882546 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:33.882588 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:33.882637 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:33.882718 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:33.882749 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:33.882779 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:33.882815 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:33.882844 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:33.914657 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:33.918042 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:33.918140 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:33.955710 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:34.135478 8834 executor.go:111] Tasks: 0 done / 118 total; 46 can run I0512 07:42:34.135823 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:34.135945 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:34.135971 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:34.136061 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0512 07:42:34.136022 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:34.136223 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:34.138857 8834 executor.go:111] Tasks: 46 done / 118 total; 23 can run I0512 07:42:34.145173 8834 executor.go:111] Tasks: 69 done / 118 total; 34 can run I0512 07:42:34.145599 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:34.145779 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:34.151293 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:34.152469 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:34.152610 8834 executor.go:111] Tasks: 103 done / 118 total; 9 can run I0512 07:42:34.153167 8834 executor.go:111] Tasks: 112 done / 118 total; 6 can run I0512 07:42:34.153764 8834 executor.go:111] Tasks: 118 done / 118 total; 0 can run I0512 07:42:34.223450 8834 target.go:112] Terraform output is in /tmp/test306516611/out --- PASS: TestMinimalIPv6Private (0.35s) === RUN TestMinimalIPv6Calico I0512 07:42:34.228281 8834 featureflag.go:162] FeatureFlag "AWSIPv6"=true I0512 07:42:34.228541 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:34.228600 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:34.228655 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:34.228708 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:34.228806 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:34.228840 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:34.228876 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:34.228913 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:34.228942 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:34.258926 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:34.262149 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:34.262253 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:34.300242 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:34.709403 8834 executor.go:111] Tasks: 0 done / 107 total; 47 can run I0512 07:42:34.711258 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:34.711995 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:34.712054 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:34.712159 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0512 07:42:34.712825 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:34.712835 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:34.717535 8834 executor.go:111] Tasks: 47 done / 107 total; 21 can run I0512 07:42:34.722779 8834 executor.go:111] Tasks: 68 done / 107 total; 31 can run I0512 07:42:34.723354 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:34.723591 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:34.724698 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:34.726768 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:34.726913 8834 executor.go:111] Tasks: 99 done / 107 total; 5 can run I0512 07:42:34.727399 8834 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0512 07:42:34.727745 8834 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0512 07:42:34.794515 8834 target.go:112] Terraform output is in /tmp/test2415744387/out I0512 07:42:34.800253 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:34.800318 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:34.800369 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:34.800423 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:34.800468 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:34.800499 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:34.800530 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:34.800567 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:34.800598 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:34.830248 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:34.834069 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:34.834173 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:34.871891 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0512 07:42:35.275522 8834 executor.go:111] Tasks: 0 done / 107 total; 47 can run I0512 07:42:35.276234 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:35.276412 8834 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0512 07:42:35.276437 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:35.277418 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } W0512 07:42:35.277755 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:35.276363 8834 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0512 07:42:35.278045 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:35.278195 8834 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0512 07:42:35.276461 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:35.283171 8834 executor.go:111] Tasks: 47 done / 107 total; 21 can run I0512 07:42:35.288754 8834 executor.go:111] Tasks: 68 done / 107 total; 31 can run I0512 07:42:35.289242 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:35.289502 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:35.290476 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:35.292147 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:35.292303 8834 executor.go:111] Tasks: 99 done / 107 total; 5 can run I0512 07:42:35.292731 8834 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0512 07:42:35.293052 8834 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0512 07:42:35.297104 8834 target.go:168] Cloudformation output is in /tmp/test4286997798/out I0512 07:42:35.309369 8834 featureflag.go:162] FeatureFlag "AWSIPv6"=false --- PASS: TestMinimalIPv6Calico (1.08s) === RUN TestMinimalIPv6Cilium I0512 07:42:35.309552 8834 featureflag.go:162] FeatureFlag "AWSIPv6"=true I0512 07:42:35.309993 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:35.310121 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:35.310219 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:35.310324 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:35.311037 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:35.311830 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:35.311894 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:35.311933 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:35.311961 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:35.338200 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:35.341685 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:35.341782 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:35.379458 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:35.597167 8834 executor.go:111] Tasks: 0 done / 106 total; 47 can run W0512 07:42:35.597392 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:35.599133 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:35.597555 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:35.601797 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:35.601949 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0512 07:42:35.601907 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:35.605140 8834 executor.go:111] Tasks: 47 done / 106 total; 21 can run I0512 07:42:35.613341 8834 executor.go:111] Tasks: 68 done / 106 total; 30 can run I0512 07:42:35.613901 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:35.614130 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:35.618278 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:35.619756 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:35.619919 8834 executor.go:111] Tasks: 98 done / 106 total; 5 can run I0512 07:42:35.620501 8834 executor.go:111] Tasks: 103 done / 106 total; 3 can run I0512 07:42:35.620819 8834 executor.go:111] Tasks: 106 done / 106 total; 0 can run I0512 07:42:35.684241 8834 target.go:112] Terraform output is in /tmp/test570697328/out I0512 07:42:35.689476 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:35.689541 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:35.689595 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:35.689651 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:35.689697 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:35.689728 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:35.689762 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:35.689799 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:35.689829 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:35.715569 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:35.718962 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:35.719053 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:35.756194 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0512 07:42:35.978610 8834 executor.go:111] Tasks: 0 done / 106 total; 47 can run I0512 07:42:35.979340 8834 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0512 07:42:35.979380 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:35.979465 8834 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0512 07:42:35.979352 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0512 07:42:35.979673 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:35.979655 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:35.979861 8834 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0512 07:42:35.979883 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:35.979991 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0512 07:42:35.981986 8834 executor.go:111] Tasks: 47 done / 106 total; 21 can run I0512 07:42:35.989925 8834 executor.go:111] Tasks: 68 done / 106 total; 30 can run I0512 07:42:35.990269 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:35.990706 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:35.992066 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:35.993003 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:35.993160 8834 executor.go:111] Tasks: 98 done / 106 total; 5 can run I0512 07:42:35.993592 8834 executor.go:111] Tasks: 103 done / 106 total; 3 can run I0512 07:42:35.993903 8834 executor.go:111] Tasks: 106 done / 106 total; 0 can run I0512 07:42:35.998155 8834 target.go:168] Cloudformation output is in /tmp/test528285165/out I0512 07:42:36.010205 8834 featureflag.go:162] FeatureFlag "AWSIPv6"=false --- PASS: TestMinimalIPv6Cilium (0.70s) === RUN TestMinimalWarmPool I0512 07:42:36.010535 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:36.010586 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:36.010630 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:36.010680 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:36.010761 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:36.010792 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:36.010823 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:36.010858 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:36.010886 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:36.035982 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:36.039311 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:36.039406 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:36.078204 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:36.277866 8834 executor.go:111] Tasks: 0 done / 89 total; 46 can run I0512 07:42:36.278299 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:36.278332 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:36.278449 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0512 07:42:36.278453 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:36.278579 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:36.278616 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-warmpool.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:36.284567 8834 executor.go:111] Tasks: 46 done / 89 total; 19 can run I0512 07:42:36.287516 8834 executor.go:111] Tasks: 65 done / 89 total; 21 can run I0512 07:42:36.287791 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:36.288063 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:36.289305 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:36.289415 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:36.289484 8834 executor.go:111] Tasks: 86 done / 89 total; 2 can run I0512 07:42:36.289823 8834 executor.go:111] Tasks: 88 done / 89 total; 1 can run I0512 07:42:36.290217 8834 executor.go:111] Tasks: 89 done / 89 total; 0 can run I0512 07:42:36.340752 8834 target.go:112] Terraform output is in /tmp/test3280343235/out --- PASS: TestMinimalWarmPool (0.33s) === RUN TestMinimalEtcd I0512 07:42:36.345509 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:36.345579 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:36.345629 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:36.345682 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:36.345770 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:36.345805 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:36.345836 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:36.345873 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:36.345904 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:36.371868 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:36.374961 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:36.375062 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:36.411823 8834 zones.go:93] ListHostedZonesPages { } W0512 07:42:36.501955 8834 model.go:247] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W0512 07:42:36.502021 8834 model.go:347] overriding log level in manifest (embedded etcd manifest), new level is 10 W0512 07:42:36.503951 8834 model.go:247] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W0512 07:42:36.504118 8834 model.go:511] overloading ENV var in manifest (embedded etcd manifest) with ETCD_MANAGER_HOURLY_BACKUPS_RETENTION=1d W0512 07:42:36.504158 8834 model.go:511] overloading ENV var in manifest (embedded etcd manifest) with ETCD_MANAGER_DAILY_BACKUPS_RETENTION=30d ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0512 07:42:36.516297 8834 executor.go:111] Tasks: 0 done / 86 total; 44 can run W0512 07:42:36.516964 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0512 07:42:36.517017 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:36.517339 8834 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0512 07:42:36.517324 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:36.517377 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0512 07:42:36.517458 8834 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0512 07:42:36.517473 8834 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0512 07:42:36.517477 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:36.517581 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-etcd.example.com" },{ Key: "KubernetesCluster", Value: "minimal-etcd.example.com" },{ Key: "kubernetes.io/cluster/minimal-etcd.example.com", Value: "owned" }] }] } I0512 07:42:36.519654 8834 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0512 07:42:36.527996 8834 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0512 07:42:36.528341 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:36.528623 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:36.529718 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:36.530988 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:36.531130 8834 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0512 07:42:36.531407 8834 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0512 07:42:36.534354 8834 target.go:168] Cloudformation output is in /tmp/test2602061632/out --- PASS: TestMinimalEtcd (0.20s) === RUN TestMinimalGp3 I0512 07:42:36.545324 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:36.545375 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:36.545418 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:36.545465 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:36.545546 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:36.545576 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:36.545606 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:36.545642 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:36.545669 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:36.570411 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:36.573311 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:36.573400 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:36.611786 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:36.718881 8834 executor.go:111] Tasks: 0 done / 86 total; 44 can run W0512 07:42:36.720322 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:36.720600 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:36.720630 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:36.720748 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0512 07:42:36.720980 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:36.721559 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:36.725680 8834 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0512 07:42:36.730742 8834 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0512 07:42:36.731083 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:36.731336 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:36.732581 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:36.733505 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:36.733616 8834 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0512 07:42:36.733965 8834 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0512 07:42:36.788651 8834 target.go:112] Terraform output is in /tmp/test615000829/out I0512 07:42:36.793162 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:36.793239 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:36.793322 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:36.793410 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:36.793471 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:36.793529 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:36.793586 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:36.793651 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:36.793698 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:36.818990 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:36.822072 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:36.822198 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:36.861135 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0512 07:42:36.966256 8834 executor.go:111] Tasks: 0 done / 86 total; 44 can run W0512 07:42:36.967057 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:36.967051 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:36.967188 8834 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0512 07:42:36.967207 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:36.967291 8834 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0512 07:42:36.967058 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:36.967301 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0512 07:42:36.967323 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:36.967415 8834 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0512 07:42:36.970905 8834 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0512 07:42:36.978638 8834 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0512 07:42:36.978879 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:36.979239 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:36.979927 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:36.981665 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:36.981804 8834 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0512 07:42:36.982090 8834 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0512 07:42:36.985321 8834 target.go:168] Cloudformation output is in /tmp/test2368123961/out --- PASS: TestMinimalGp3 (0.45s) === RUN TestExistingIAMCloudformation I0512 07:42:36.996415 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:36.996465 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:36.996508 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:36.996562 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:36.996640 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:36.996671 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:36.996702 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:36.996737 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:36.996766 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:37.024381 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:37.027192 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.027292 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.067151 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0512 07:42:37.172316 8834 executor.go:111] Tasks: 0 done / 76 total; 42 can run W0512 07:42:37.172900 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0512 07:42:37.172911 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:37.173119 8834 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0512 07:42:37.173212 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:37.173163 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:37.174927 8834 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0512 07:42:37.174946 8834 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0512 07:42:37.174979 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:37.175143 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0512 07:42:37.179463 8834 executor.go:111] Tasks: 42 done / 76 total; 11 can run I0512 07:42:37.184667 8834 executor.go:111] Tasks: 53 done / 76 total; 21 can run I0512 07:42:37.185018 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.185165 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.186344 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.187303 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.187435 8834 executor.go:111] Tasks: 74 done / 76 total; 2 can run I0512 07:42:37.187716 8834 executor.go:111] Tasks: 76 done / 76 total; 0 can run I0512 07:42:37.190120 8834 target.go:168] Cloudformation output is in /tmp/test2414470748/out --- PASS: TestExistingIAMCloudformation (0.20s) === RUN TestExistingSG I0512 07:42:37.199033 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:37.199083 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:37.199128 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:37.199174 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:37.199218 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:37.199288 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:37.199326 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:37.199357 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:37.199386 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:37.236021 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:37.239294 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.239402 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.239488 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.239573 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.277056 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:37.386769 8834 executor.go:111] Tasks: 0 done / 139 total; 50 can run W0512 07:42:37.387531 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:37.389169 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existingsg.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:37.389433 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:37.389482 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:37.389623 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0512 07:42:37.389892 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:37.390896 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:37.391368 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:37.394424 8834 executor.go:111] Tasks: 50 done / 139 total; 26 can run I0512 07:42:37.399895 8834 executor.go:111] Tasks: 76 done / 139 total; 58 can run I0512 07:42:37.400213 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.400546 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.401889 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.402815 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.403510 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.406957 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.407957 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.408084 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.408203 8834 executor.go:111] Tasks: 134 done / 139 total; 5 can run I0512 07:42:37.408602 8834 executor.go:111] Tasks: 139 done / 139 total; 0 can run I0512 07:42:37.500494 8834 target.go:112] Terraform output is in /tmp/test4091175379/out --- PASS: TestExistingSG (0.31s) === RUN TestBastionAdditionalUserData I0512 07:42:37.505669 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:37.505726 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:37.505775 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:37.505827 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:37.505872 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:37.505902 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:37.505934 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:37.505970 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:37.506000 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:37.537683 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:37.541470 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.541593 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.541660 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.578851 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:37.689521 8834 executor.go:111] Tasks: 0 done / 120 total; 49 can run W0512 07:42:37.689984 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:37.690035 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:37.690444 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.bastionuserdata.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:37.690534 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:37.690573 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0512 07:42:37.690673 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:37.690705 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0512 07:42:37.696549 8834 executor.go:111] Tasks: 49 done / 120 total; 29 can run I0512 07:42:37.702034 8834 executor.go:111] Tasks: 78 done / 120 total; 36 can run I0512 07:42:37.702268 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.702593 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.702927 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.703153 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.703340 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.705374 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.705494 8834 executor.go:111] Tasks: 114 done / 120 total; 5 can run I0512 07:42:37.705919 8834 executor.go:111] Tasks: 119 done / 120 total; 1 can run I0512 07:42:37.706087 8834 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0512 07:42:37.786246 8834 target.go:112] Terraform output is in /tmp/test3609860052/out --- PASS: TestBastionAdditionalUserData (0.29s) === RUN TestPrivateWeave I0512 07:42:37.791083 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:37.791163 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:37.791218 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:37.791274 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:37.791369 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:37.791399 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:37.791430 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:37.791467 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:37.791496 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:37.819715 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:37.822654 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.822743 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.822802 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:37.862157 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:37.992058 8834 executor.go:111] Tasks: 0 done / 118 total; 48 can run I0512 07:42:37.992527 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0512 07:42:37.992626 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:37.992554 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:37.992689 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0512 07:42:37.993030 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:37.993125 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateweave.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:37.993259 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:37.999254 8834 executor.go:111] Tasks: 48 done / 118 total; 28 can run I0512 07:42:38.004436 8834 executor.go:111] Tasks: 76 done / 118 total; 36 can run I0512 07:42:38.005100 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:38.005366 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:38.005458 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:38.005652 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:38.007308 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:38.008138 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:38.008272 8834 executor.go:111] Tasks: 112 done / 118 total; 5 can run I0512 07:42:38.008795 8834 executor.go:111] Tasks: 117 done / 118 total; 1 can run I0512 07:42:38.009138 8834 executor.go:111] Tasks: 118 done / 118 total; 0 can run I0512 07:42:38.086734 8834 target.go:112] Terraform output is in /tmp/test3562044889/out --- PASS: TestPrivateWeave (0.30s) === RUN TestPrivateFlannel I0512 07:42:38.091475 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:38.091530 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:38.091582 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:38.091633 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:38.091723 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:38.091754 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:38.091786 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:38.091821 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:38.091849 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:38.119358 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:38.122299 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:38.122393 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:38.122458 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:38.159343 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:38.254556 8834 executor.go:111] Tasks: 0 done / 118 total; 48 can run W0512 07:42:38.254884 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:38.254940 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:38.255147 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:38.255167 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:38.255221 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0512 07:42:38.255256 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:38.255297 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateflannel.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:38.256221 8834 executor.go:111] Tasks: 48 done / 118 total; 28 can run I0512 07:42:38.259927 8834 executor.go:111] Tasks: 76 done / 118 total; 36 can run I0512 07:42:38.260302 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:38.260474 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:38.260657 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:38.260822 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:38.261420 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:38.261957 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:38.262041 8834 executor.go:111] Tasks: 112 done / 118 total; 5 can run I0512 07:42:38.262511 8834 executor.go:111] Tasks: 117 done / 118 total; 1 can run I0512 07:42:38.262677 8834 executor.go:111] Tasks: 118 done / 118 total; 0 can run I0512 07:42:38.340529 8834 target.go:112] Terraform output is in /tmp/test3122606595/out --- PASS: TestPrivateFlannel (0.25s) === RUN TestPrivateCalico I0512 07:42:38.345391 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:38.345449 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:38.345501 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:38.345553 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:38.345640 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:38.345671 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:38.345703 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:38.345737 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:38.345764 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:38.375612 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:38.379278 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:38.379434 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:38.379549 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:38.417451 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:38.706466 8834 executor.go:111] Tasks: 0 done / 119 total; 48 can run W0512 07:42:38.706732 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:38.706956 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:38.707050 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:38.709506 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:38.709543 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:38.709736 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0512 07:42:38.710220 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:38.713359 8834 executor.go:111] Tasks: 48 done / 119 total; 28 can run I0512 07:42:38.716808 8834 executor.go:111] Tasks: 76 done / 119 total; 37 can run I0512 07:42:38.717215 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:38.717418 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:38.717620 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:38.717861 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:38.718394 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:38.719948 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:38.720070 8834 executor.go:111] Tasks: 113 done / 119 total; 5 can run I0512 07:42:38.720529 8834 executor.go:111] Tasks: 118 done / 119 total; 1 can run I0512 07:42:38.720780 8834 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0512 07:42:38.778914 8834 target.go:112] Terraform output is in /tmp/test982143074/out I0512 07:42:38.784251 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:38.784315 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:38.784366 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:38.784421 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:38.784464 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:38.784511 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:38.784541 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:38.784578 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:38.784607 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:38.812110 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:38.815035 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:38.815122 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:38.815182 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:38.854327 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0512 07:42:39.188579 8834 executor.go:111] Tasks: 0 done / 119 total; 48 can run I0512 07:42:39.188778 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:39.188874 8834 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0512 07:42:39.188886 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:39.188953 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } W0512 07:42:39.190971 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:39.191177 8834 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0512 07:42:39.191212 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:39.191345 8834 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0512 07:42:39.191964 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0512 07:42:39.192128 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:39.195208 8834 executor.go:111] Tasks: 48 done / 119 total; 28 can run I0512 07:42:39.200552 8834 executor.go:111] Tasks: 76 done / 119 total; 37 can run I0512 07:42:39.201067 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:39.201238 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:39.201381 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:39.201477 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:39.206233 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:39.207578 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:39.207744 8834 executor.go:111] Tasks: 113 done / 119 total; 5 can run I0512 07:42:39.208167 8834 executor.go:111] Tasks: 118 done / 119 total; 1 can run I0512 07:42:39.208357 8834 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0512 07:42:39.213872 8834 target.go:168] Cloudformation output is in /tmp/test3797808952/out --- PASS: TestPrivateCalico (0.88s) === RUN TestPrivateCilium I0512 07:42:39.227566 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:39.227616 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:39.227663 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:39.227714 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:39.227793 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:39.227856 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:39.227888 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:39.227922 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:39.227950 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:39.256302 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:39.260904 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:39.261004 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:39.261073 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:39.297683 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:39.451647 8834 executor.go:111] Tasks: 0 done / 118 total; 48 can run W0512 07:42:39.452001 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:39.452425 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:39.454116 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:39.452492 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:39.454328 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:39.454425 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0512 07:42:39.452580 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:39.459822 8834 executor.go:111] Tasks: 48 done / 118 total; 28 can run I0512 07:42:39.465208 8834 executor.go:111] Tasks: 76 done / 118 total; 36 can run I0512 07:42:39.465452 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:39.465570 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:39.465913 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:39.466163 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:39.466909 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:39.467791 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:39.467941 8834 executor.go:111] Tasks: 112 done / 118 total; 5 can run I0512 07:42:39.468498 8834 executor.go:111] Tasks: 117 done / 118 total; 1 can run I0512 07:42:39.468727 8834 executor.go:111] Tasks: 118 done / 118 total; 0 can run I0512 07:42:39.548816 8834 target.go:112] Terraform output is in /tmp/test2501267675/out I0512 07:42:39.555906 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:39.556202 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:39.556278 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:39.556340 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:39.556567 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:39.556621 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:39.556659 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:39.556700 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:39.556732 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:39.586181 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:39.589369 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:39.589467 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:39.589531 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:39.627435 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0512 07:42:39.781155 8834 executor.go:111] Tasks: 0 done / 118 total; 48 can run W0512 07:42:39.781400 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0512 07:42:39.781606 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:39.781739 8834 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0512 07:42:39.781768 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:39.781895 8834 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0512 07:42:39.782152 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:39.782264 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0512 07:42:39.782347 8834 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0512 07:42:39.782377 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:39.782507 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecilium.example.com" },{ Key: "KubernetesCluster", Value: "privatecilium.example.com" },{ Key: "kubernetes.io/cluster/privatecilium.example.com", Value: "owned" }] }] } I0512 07:42:39.783945 8834 executor.go:111] Tasks: 48 done / 118 total; 28 can run I0512 07:42:39.793405 8834 executor.go:111] Tasks: 76 done / 118 total; 36 can run I0512 07:42:39.793927 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:39.794117 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:39.794239 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:39.794413 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:39.795555 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:39.797132 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:39.797272 8834 executor.go:111] Tasks: 112 done / 118 total; 5 can run I0512 07:42:39.797781 8834 executor.go:111] Tasks: 117 done / 118 total; 1 can run I0512 07:42:39.798012 8834 executor.go:111] Tasks: 118 done / 118 total; 0 can run I0512 07:42:39.802506 8834 target.go:168] Cloudformation output is in /tmp/test177907193/out --- PASS: TestPrivateCilium (0.59s) === RUN TestPrivateCilium2 I0512 07:42:39.816192 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:39.816246 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:39.816291 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:39.816355 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:39.816433 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:39.816464 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:39.816496 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:39.816529 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:39.816557 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:39.849175 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.18.20 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.18.20 ********************************************************************************* I0512 07:42:39.852444 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:39.852540 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:39.852603 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:41.303304 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:41.450834 8834 executor.go:111] Tasks: 0 done / 121 total; 49 can run W0512 07:42:41.451285 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:41.451303 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:41.451485 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:41.451522 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:41.451549 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:41.451648 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0512 07:42:41.451842 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:41.454460 8834 executor.go:111] Tasks: 49 done / 121 total; 29 can run I0512 07:42:41.459232 8834 executor.go:111] Tasks: 78 done / 121 total; 35 can run I0512 07:42:41.459495 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:41.459667 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:41.459745 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:41.462015 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:41.462138 8834 executor.go:111] Tasks: 113 done / 121 total; 6 can run I0512 07:42:41.462399 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:41.464541 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:41.464783 8834 executor.go:111] Tasks: 119 done / 121 total; 2 can run I0512 07:42:41.465451 8834 executor.go:111] Tasks: 121 done / 121 total; 0 can run I0512 07:42:41.544405 8834 target.go:112] Terraform output is in /tmp/test946250086/out I0512 07:42:41.549580 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:41.549646 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:41.549699 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:41.549756 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:41.549798 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:41.549826 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:41.549857 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:41.549891 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:41.549921 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:41.586970 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.18.20 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.18.20 ********************************************************************************* I0512 07:42:41.593062 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:41.593214 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:41.593321 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:41.636114 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0512 07:42:41.776720 8834 executor.go:111] Tasks: 0 done / 121 total; 49 can run W0512 07:42:41.780064 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0512 07:42:41.780463 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:41.781102 8834 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0512 07:42:41.781241 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:41.781202 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:41.781499 8834 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0512 07:42:41.782804 8834 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key W0512 07:42:41.781742 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:41.782841 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:41.782975 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecilium.example.com" },{ Key: "KubernetesCluster", Value: "privatecilium.example.com" },{ Key: "kubernetes.io/cluster/privatecilium.example.com", Value: "owned" }] }] } I0512 07:42:41.784516 8834 executor.go:111] Tasks: 49 done / 121 total; 29 can run I0512 07:42:41.789729 8834 executor.go:111] Tasks: 78 done / 121 total; 35 can run I0512 07:42:41.790145 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:41.790446 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:41.790615 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:41.792750 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:41.792983 8834 executor.go:111] Tasks: 113 done / 121 total; 6 can run I0512 07:42:41.793373 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:41.794396 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:41.794528 8834 executor.go:111] Tasks: 119 done / 121 total; 2 can run I0512 07:42:41.794754 8834 executor.go:111] Tasks: 121 done / 121 total; 0 can run I0512 07:42:41.799821 8834 target.go:168] Cloudformation output is in /tmp/test3856233230/out --- PASS: TestPrivateCilium2 (2.00s) === RUN TestPrivateCiliumAdvanced I0512 07:42:41.815748 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:41.815799 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:41.815843 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:41.815897 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:41.815976 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:41.816017 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:41.816057 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:41.816094 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:41.816122 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:41.848728 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:41.851784 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:41.851863 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:41.851918 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:41.889390 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:42.050065 8834 executor.go:111] Tasks: 0 done / 124 total; 54 can run W0512 07:42:42.050554 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:42.050584 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:42.050779 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:42.050929 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:42.050961 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:42.051055 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0512 07:42:42.051110 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:42.053746 8834 executor.go:111] Tasks: 54 done / 124 total; 28 can run I0512 07:42:42.065812 8834 executor.go:111] Tasks: 82 done / 124 total; 36 can run I0512 07:42:42.066302 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:42.066529 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:42.066746 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:42.067008 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:42.068083 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:42.069176 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:42.069301 8834 executor.go:111] Tasks: 118 done / 124 total; 5 can run I0512 07:42:42.069835 8834 executor.go:111] Tasks: 123 done / 124 total; 1 can run I0512 07:42:42.070120 8834 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0512 07:42:42.152647 8834 target.go:112] Terraform output is in /tmp/test2411010378/out I0512 07:42:42.157587 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:42.157652 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:42.157707 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:42.157764 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:42.157823 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:42.157855 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:42.157887 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:42.157923 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:42.157952 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:42.191114 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:42.194563 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:42.194728 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:42.194846 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:42.232525 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0512 07:42:42.390296 8834 executor.go:111] Tasks: 0 done / 124 total; 54 can run W0512 07:42:42.391263 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:42.391287 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:42.391375 8834 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0512 07:42:42.391414 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0512 07:42:42.391472 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0512 07:42:42.391487 8834 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0512 07:42:42.391404 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:42.391608 8834 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0512 07:42:42.391527 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:42.391665 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privateciliumadvanced.example.com" },{ Key: "KubernetesCluster", Value: "privateciliumadvanced.example.com" },{ Key: "kubernetes.io/cluster/privateciliumadvanced.example.com", Value: "owned" }] }] } I0512 07:42:42.394021 8834 executor.go:111] Tasks: 54 done / 124 total; 28 can run I0512 07:42:42.399608 8834 executor.go:111] Tasks: 82 done / 124 total; 36 can run I0512 07:42:42.399985 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:42.401245 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:42.401729 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:42.401958 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:42.405791 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:42.406732 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:42.406893 8834 executor.go:111] Tasks: 118 done / 124 total; 5 can run I0512 07:42:42.407281 8834 executor.go:111] Tasks: 123 done / 124 total; 1 can run I0512 07:42:42.407429 8834 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0512 07:42:42.411659 8834 target.go:168] Cloudformation output is in /tmp/test1198626272/out --- PASS: TestPrivateCiliumAdvanced (0.61s) === RUN TestPrivateCanal I0512 07:42:42.426979 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:42.427033 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:42.427081 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:42.427135 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:42.427216 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:42.427245 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:42.427275 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:42.427308 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:42.427337 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:42.454586 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.4 ********************************************************************************* I0512 07:42:42.457213 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:42.457296 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:42.457354 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:43.434971 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:43.836574 8834 executor.go:111] Tasks: 0 done / 119 total; 49 can run W0512 07:42:43.837020 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:43.837328 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecanal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:43.837427 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:43.837484 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:43.837673 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:43.837696 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:43.837763 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0512 07:42:43.840171 8834 executor.go:111] Tasks: 49 done / 119 total; 28 can run I0512 07:42:43.845626 8834 executor.go:111] Tasks: 77 done / 119 total; 36 can run I0512 07:42:43.846317 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:43.846514 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:43.846598 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:43.846788 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:43.847978 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:43.849306 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:43.849431 8834 executor.go:111] Tasks: 113 done / 119 total; 5 can run I0512 07:42:43.850657 8834 executor.go:111] Tasks: 118 done / 119 total; 1 can run I0512 07:42:43.851215 8834 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0512 07:42:43.930737 8834 target.go:112] Terraform output is in /tmp/test3928610213/out --- PASS: TestPrivateCanal (1.51s) === RUN TestPrivateKopeio I0512 07:42:43.936313 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:43.936374 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:43.936425 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:43.936479 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:43.936573 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:43.936604 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:43.936635 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:43.936672 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:43.936699 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:43.969527 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:43.973103 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:43.973278 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:43.973395 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.010783 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:44.132248 8834 executor.go:111] Tasks: 0 done / 124 total; 48 can run I0512 07:42:44.132582 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:44.132615 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:44.132712 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0512 07:42:44.132962 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:44.132974 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:44.132964 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:44.133195 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:44.136481 8834 executor.go:111] Tasks: 48 done / 124 total; 31 can run I0512 07:42:44.145052 8834 executor.go:111] Tasks: 79 done / 124 total; 39 can run I0512 07:42:44.145492 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.145679 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.145854 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.145939 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.146608 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.148528 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.148654 8834 executor.go:111] Tasks: 118 done / 124 total; 6 can run I0512 07:42:44.149082 8834 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0512 07:42:44.228818 8834 target.go:112] Terraform output is in /tmp/test2961051554/out --- PASS: TestPrivateKopeio (0.30s) === RUN TestUnmanaged I0512 07:42:44.233712 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:44.233772 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:44.233843 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:44.233900 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:44.233946 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:44.233976 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:44.234008 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:44.234044 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:44.234074 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:44.267826 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:44.270929 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.271031 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.271098 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.309529 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:44.422253 8834 executor.go:111] Tasks: 0 done / 106 total; 46 can run W0512 07:42:44.422627 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:44.422884 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:44.422916 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:44.423016 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0512 07:42:44.423152 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:44.423250 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.unmanaged.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:44.423364 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:44.426106 8834 executor.go:111] Tasks: 46 done / 106 total; 25 can run I0512 07:42:44.435225 8834 executor.go:111] Tasks: 71 done / 106 total; 31 can run I0512 07:42:44.435750 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.435942 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.436152 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.436237 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.436797 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.438616 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.438762 8834 executor.go:111] Tasks: 102 done / 106 total; 4 can run I0512 07:42:44.439243 8834 executor.go:111] Tasks: 106 done / 106 total; 0 can run I0512 07:42:44.511492 8834 target.go:112] Terraform output is in /tmp/test594810165/out --- PASS: TestUnmanaged (0.28s) === RUN TestPrivateSharedSubnet I0512 07:42:44.516298 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:44.516357 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:44.516410 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:44.516464 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:44.516506 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:44.516535 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:44.516569 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:44.516605 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:44.516636 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:44.545866 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:44.549337 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.549431 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.549497 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.589497 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:44.698417 8834 executor.go:111] Tasks: 0 done / 105 total; 46 can run W0512 07:42:44.698807 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:44.699148 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:44.699184 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:44.699308 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:44.699478 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:44.699519 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:44.699641 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0512 07:42:44.705957 8834 executor.go:111] Tasks: 46 done / 105 total; 24 can run I0512 07:42:44.706652 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0512 07:42:44.711385 8834 executor.go:111] Tasks: 70 done / 105 total; 31 can run I0512 07:42:44.711989 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.712195 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.712261 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.712533 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.713610 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.714640 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.714782 8834 executor.go:111] Tasks: 101 done / 105 total; 4 can run I0512 07:42:44.715407 8834 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0512 07:42:44.786656 8834 target.go:112] Terraform output is in /tmp/test1697914570/out --- PASS: TestPrivateSharedSubnet (0.27s) === RUN TestPrivateSharedIP I0512 07:42:44.791329 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:44.791386 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:44.791436 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:44.791492 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:44.791578 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:44.791607 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:44.791638 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:44.791677 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:44.791707 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:44.820121 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:44.823042 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.823128 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.823188 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.860704 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:44.961417 8834 executor.go:111] Tasks: 0 done / 114 total; 46 can run W0512 07:42:44.961877 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:44.962043 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:44.962163 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:44.962328 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:44.962631 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:44.962678 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:44.962809 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0512 07:42:44.967917 8834 executor.go:111] Tasks: 46 done / 114 total; 26 can run I0512 07:42:44.968499 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0512 07:42:44.973065 8834 executor.go:111] Tasks: 72 done / 114 total; 36 can run I0512 07:42:44.973318 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.973653 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.973851 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.974109 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.974539 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.975919 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:44.976041 8834 executor.go:111] Tasks: 108 done / 114 total; 5 can run I0512 07:42:44.976489 8834 executor.go:111] Tasks: 113 done / 114 total; 1 can run I0512 07:42:44.976663 8834 executor.go:111] Tasks: 114 done / 114 total; 0 can run I0512 07:42:45.052416 8834 target.go:112] Terraform output is in /tmp/test3587894408/out I0512 07:42:45.056888 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:45.056950 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:45.057002 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:45.057055 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:45.057097 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:45.057128 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:45.057161 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:45.057198 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:45.057227 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:45.085097 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:45.088136 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:45.088222 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:45.088282 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:45.125176 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0512 07:42:45.212994 8834 executor.go:111] Tasks: 0 done / 114 total; 46 can run W0512 07:42:45.213404 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0512 07:42:45.213470 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0512 07:42:45.213466 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:45.213650 8834 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0512 07:42:45.213676 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:45.213757 8834 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0512 07:42:45.213792 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:45.213911 8834 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0512 07:42:45.213926 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:45.214019 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0512 07:42:45.215122 8834 executor.go:111] Tasks: 46 done / 114 total; 26 can run I0512 07:42:45.215473 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0512 07:42:45.219258 8834 executor.go:111] Tasks: 72 done / 114 total; 36 can run I0512 07:42:45.219588 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:45.219699 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:45.219867 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:45.220072 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:45.220544 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:45.222426 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:45.222553 8834 executor.go:111] Tasks: 108 done / 114 total; 5 can run I0512 07:42:45.222910 8834 executor.go:111] Tasks: 113 done / 114 total; 1 can run I0512 07:42:45.223044 8834 executor.go:111] Tasks: 114 done / 114 total; 0 can run I0512 07:42:45.227575 8834 target.go:168] Cloudformation output is in /tmp/test1548136072/out --- PASS: TestPrivateSharedIP (0.44s) === RUN TestPrivateDns1 I0512 07:42:45.235339 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:45.235380 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:45.235412 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:45.235451 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:45.235509 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:45.235528 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:45.235546 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:45.235576 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:45.235594 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:45.254282 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:45.254337 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:45.254376 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:45.413424 8834 executor.go:111] Tasks: 0 done / 118 total; 47 can run W0512 07:42:45.414042 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:45.414181 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns1.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:45.414289 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:45.414378 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:45.415871 8834 executor.go:111] Tasks: 47 done / 118 total; 24 can run I0512 07:42:45.416252 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "internal.example.com" I0512 07:42:45.416278 8834 zones.go:40] GetHostedZone { Id: "/hostedzone/Z2AFAKE1ZON3NO" } I0512 07:42:45.416331 8834 dnszone.go:249] Existing zone "internal.example.com." found; will configure TF to reuse I0512 07:42:45.416341 8834 dnszone.go:272] No association between VPC "privatedns1.example.com" and zone "internal.example.com."; adding I0512 07:42:45.416391 8834 executor.go:111] Tasks: 71 done / 118 total; 35 can run I0512 07:42:45.416930 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:45.417050 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:45.419681 8834 executor.go:111] Tasks: 106 done / 118 total; 5 can run W0512 07:42:45.419796 8834 api.go:53] PageSize not implemented W0512 07:42:45.419866 8834 api.go:53] PageSize not implemented W0512 07:42:45.419839 8834 bootstrapscript.go:91] Task did not have an address: *awstasks.ClassicLoadBalancer {"Name":"api.privatedns1.example.com","Lifecycle":"Sync","LoadBalancerName":"api-privatedns1-example-c-lq96ht","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-us-test-1a.privatedns1.example.com","ShortName":"utility-us-test-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"privatedns1.example.com","Lifecycle":"Sync","ID":null,"CIDR":"172.20.0.0/16","AmazonIPv6":true,"IPv6CIDR":null,"EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"AssociateExtraCIDRBlocks":null},"AmazonIPv6CIDR":null,"AvailabilityZone":"us-test-1a","CIDR":"172.20.4.0/22","IPv6CIDR":null,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"utility-us-test-1a.privatedns1.example.com","Owner":"John Doe","SubnetType":"Utility","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.privatedns1.example.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"privatedns1.example.com","Lifecycle":"Sync","ID":null,"CIDR":"172.20.0.0/16","AmazonIPv6":true,"IPv6CIDR":null,"EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"AssociateExtraCIDRBlocks":null},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"api-elb.privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":{"EmitInterval":null,"Enabled":false,"S3BucketName":null,"S3BucketPrefix":null},"ConnectionDraining":{"Enabled":true,"Timeout":300},"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":{"Enabled":false},"SSLCertificateID":"","Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"api.privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"ForAPIServer":true,"Shared":null} W0512 07:42:45.419888 8834 bootstrapscript.go:91] Task did not have an address: *awstasks.ClassicLoadBalancer {"Name":"api.privatedns1.example.com","Lifecycle":"Sync","LoadBalancerName":"api-privatedns1-example-c-lq96ht","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-us-test-1a.privatedns1.example.com","ShortName":"utility-us-test-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"privatedns1.example.com","Lifecycle":"Sync","ID":null,"CIDR":"172.20.0.0/16","AmazonIPv6":true,"IPv6CIDR":null,"EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"AssociateExtraCIDRBlocks":null},"AmazonIPv6CIDR":null,"AvailabilityZone":"us-test-1a","CIDR":"172.20.4.0/22","IPv6CIDR":null,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"utility-us-test-1a.privatedns1.example.com","Owner":"John Doe","SubnetType":"Utility","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.privatedns1.example.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"privatedns1.example.com","Lifecycle":"Sync","ID":null,"CIDR":"172.20.0.0/16","AmazonIPv6":true,"IPv6CIDR":null,"EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"AssociateExtraCIDRBlocks":null},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"api-elb.privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":{"EmitInterval":null,"Enabled":false,"S3BucketName":null,"S3BucketPrefix":null},"ConnectionDraining":{"Enabled":true,"Timeout":300},"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":{"Enabled":false},"SSLCertificateID":"","Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"api.privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"ForAPIServer":true,"Shared":null} I0512 07:42:45.425431 8834 executor.go:111] Tasks: 111 done / 118 total; 5 can run I0512 07:42:45.425643 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:45.425850 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:45.428095 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:45.431695 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:45.431852 8834 executor.go:111] Tasks: 116 done / 118 total; 2 can run I0512 07:42:45.432320 8834 executor.go:111] Tasks: 118 done / 118 total; 0 can run I0512 07:42:45.519064 8834 target.go:112] Terraform output is in /tmp/test1179350750/out --- PASS: TestPrivateDns1 (0.29s) === RUN TestPrivateDns2 I0512 07:42:45.523998 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:45.524054 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:45.524103 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:45.524157 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:45.524201 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:45.524231 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:45.524264 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:45.524300 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:45.524331 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:45.555406 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:45.555500 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:45.555564 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:45.694702 8834 executor.go:111] Tasks: 0 done / 114 total; 45 can run W0512 07:42:45.695479 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:45.695786 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:45.695799 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:45.695895 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns2.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:45.702096 8834 executor.go:111] Tasks: 45 done / 114 total; 22 can run I0512 07:42:45.702748 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "private.example.com" I0512 07:42:45.702797 8834 zones.go:40] GetHostedZone { Id: "/hostedzone/Z3AFAKE1ZOMORE" } I0512 07:42:45.702925 8834 dnszone.go:249] Existing zone "private.example.com." found; will configure TF to reuse I0512 07:42:45.702955 8834 dnszone.go:263] VPC "vpc-12345678" already associated with zone "private.example.com." I0512 07:42:45.702973 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0512 07:42:45.703144 8834 executor.go:111] Tasks: 67 done / 114 total; 35 can run I0512 07:42:45.703692 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:45.703928 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:45.706804 8834 executor.go:111] Tasks: 102 done / 114 total; 5 can run W0512 07:42:45.706976 8834 api.go:53] PageSize not implemented W0512 07:42:45.707039 8834 api.go:53] PageSize not implemented W0512 07:42:45.707008 8834 bootstrapscript.go:91] Task did not have an address: *awstasks.ClassicLoadBalancer {"Name":"api.privatedns2.example.com","Lifecycle":"Sync","LoadBalancerName":"api-privatedns2-example-c-6jft30","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-us-test-1a.privatedns2.example.com","ShortName":"utility-us-test-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"privatedns2.example.com","Lifecycle":"Sync","ID":"vpc-12345678","CIDR":"172.20.0.0/16","AmazonIPv6":null,"IPv6CIDR":null,"EnableDNSHostnames":null,"EnableDNSSupport":true,"Shared":true,"Tags":null,"AssociateExtraCIDRBlocks":null},"AmazonIPv6CIDR":null,"AvailabilityZone":"us-test-1a","CIDR":"172.20.4.0/22","IPv6CIDR":null,"Shared":false,"Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"utility-us-test-1a.privatedns2.example.com","SubnetType":"Utility","kubernetes.io/cluster/privatedns2.example.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.privatedns2.example.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"privatedns2.example.com","Lifecycle":"Sync","ID":"vpc-12345678","CIDR":"172.20.0.0/16","AmazonIPv6":null,"IPv6CIDR":null,"EnableDNSHostnames":null,"EnableDNSSupport":true,"Shared":true,"Tags":null,"AssociateExtraCIDRBlocks":null},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"api-elb.privatedns2.example.com","kubernetes.io/cluster/privatedns2.example.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":{"EmitInterval":null,"Enabled":false,"S3BucketName":null,"S3BucketPrefix":null},"ConnectionDraining":{"Enabled":true,"Timeout":300},"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":{"Enabled":false},"SSLCertificateID":"","Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"api.privatedns2.example.com","kubernetes.io/cluster/privatedns2.example.com":"owned"},"ForAPIServer":true,"Shared":null} W0512 07:42:45.707066 8834 bootstrapscript.go:91] Task did not have an address: *awstasks.ClassicLoadBalancer {"Name":"api.privatedns2.example.com","Lifecycle":"Sync","LoadBalancerName":"api-privatedns2-example-c-6jft30","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-us-test-1a.privatedns2.example.com","ShortName":"utility-us-test-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"privatedns2.example.com","Lifecycle":"Sync","ID":"vpc-12345678","CIDR":"172.20.0.0/16","AmazonIPv6":null,"IPv6CIDR":null,"EnableDNSHostnames":null,"EnableDNSSupport":true,"Shared":true,"Tags":null,"AssociateExtraCIDRBlocks":null},"AmazonIPv6CIDR":null,"AvailabilityZone":"us-test-1a","CIDR":"172.20.4.0/22","IPv6CIDR":null,"Shared":false,"Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"utility-us-test-1a.privatedns2.example.com","SubnetType":"Utility","kubernetes.io/cluster/privatedns2.example.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.privatedns2.example.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"privatedns2.example.com","Lifecycle":"Sync","ID":"vpc-12345678","CIDR":"172.20.0.0/16","AmazonIPv6":null,"IPv6CIDR":null,"EnableDNSHostnames":null,"EnableDNSSupport":true,"Shared":true,"Tags":null,"AssociateExtraCIDRBlocks":null},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"api-elb.privatedns2.example.com","kubernetes.io/cluster/privatedns2.example.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":{"EmitInterval":null,"Enabled":false,"S3BucketName":null,"S3BucketPrefix":null},"ConnectionDraining":{"Enabled":true,"Timeout":300},"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":{"Enabled":false},"SSLCertificateID":"","Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"api.privatedns2.example.com","kubernetes.io/cluster/privatedns2.example.com":"owned"},"ForAPIServer":true,"Shared":null} I0512 07:42:45.712248 8834 executor.go:111] Tasks: 107 done / 114 total; 5 can run I0512 07:42:45.712461 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:45.712586 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:45.713665 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:45.715611 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:45.715744 8834 executor.go:111] Tasks: 112 done / 114 total; 2 can run I0512 07:42:45.716112 8834 executor.go:111] Tasks: 114 done / 114 total; 0 can run I0512 07:42:45.791513 8834 target.go:112] Terraform output is in /tmp/test4185143538/out --- PASS: TestPrivateDns2 (0.27s) === RUN TestDiscoveryFeatureGate I0512 07:42:45.796923 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:45.796985 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:45.797035 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:45.797086 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:45.797129 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:45.797157 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:45.797189 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:45.797224 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:45.797252 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:45.826119 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.19.16 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.19.16 ********************************************************************************* I0512 07:42:45.829721 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:45.829821 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:47.076414 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:47.193329 8834 executor.go:111] Tasks: 0 done / 91 total; 47 can run W0512 07:42:47.193807 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:47.193962 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:47.194110 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:47.194209 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:47.194242 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:47.194435 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0512 07:42:47.196217 8834 executor.go:111] Tasks: 47 done / 91 total; 21 can run I0512 07:42:47.204629 8834 executor.go:111] Tasks: 68 done / 91 total; 21 can run I0512 07:42:47.204989 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:47.205259 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:47.206229 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:47.207684 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:47.207827 8834 executor.go:111] Tasks: 89 done / 91 total; 2 can run I0512 07:42:47.208164 8834 executor.go:111] Tasks: 91 done / 91 total; 0 can run I0512 07:42:47.265745 8834 target.go:112] Terraform output is in /tmp/test3337989623/out --- PASS: TestDiscoveryFeatureGate (1.47s) === RUN TestVFSServiceAccountIssuerDiscovery I0512 07:42:47.270614 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:47.270671 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:47.270719 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:47.270769 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:47.270817 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:47.270846 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:47.270881 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:47.270915 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:47.270946 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:47.295591 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:47.298860 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:47.298956 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:47.338364 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:47.443869 8834 executor.go:111] Tasks: 0 done / 89 total; 46 can run W0512 07:42:47.445656 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:47.446200 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:47.444626 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:47.446925 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:47.446955 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:47.447076 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0512 07:42:47.451554 8834 executor.go:111] Tasks: 46 done / 89 total; 20 can run I0512 07:42:47.456759 8834 executor.go:111] Tasks: 66 done / 89 total; 21 can run I0512 07:42:47.457091 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:47.457297 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:47.458545 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:47.459388 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:47.459501 8834 executor.go:111] Tasks: 87 done / 89 total; 2 can run I0512 07:42:47.459836 8834 executor.go:111] Tasks: 89 done / 89 total; 0 can run I0512 07:42:47.516326 8834 target.go:112] Terraform output is in /tmp/test1479620568/out --- PASS: TestVFSServiceAccountIssuerDiscovery (0.25s) === RUN TestAWSLBController I0512 07:42:47.521072 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:47.521131 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:47.521199 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:47.521255 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:47.521348 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:47.521378 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:47.521411 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:47.521450 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:47.521480 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:47.547120 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:47.550066 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:47.550161 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:47.588270 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:48.876997 8834 executor.go:111] Tasks: 0 done / 95 total; 50 can run W0512 07:42:48.877302 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:48.877415 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:48.877458 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:48.877436 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0512 07:42:48.877566 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:48.878848 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0512 07:42:48.893480 8834 executor.go:111] Tasks: 50 done / 95 total; 22 can run I0512 07:42:48.898691 8834 executor.go:111] Tasks: 72 done / 95 total; 21 can run I0512 07:42:48.899238 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:48.899536 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:48.900517 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:48.901939 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:48.902055 8834 executor.go:111] Tasks: 93 done / 95 total; 2 can run I0512 07:42:48.902398 8834 executor.go:111] Tasks: 95 done / 95 total; 0 can run I0512 07:42:48.961940 8834 target.go:112] Terraform output is in /tmp/test2538522703/out --- PASS: TestAWSLBController (1.46s) === RUN TestManyAddons I0512 07:42:48.983007 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:48.983077 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:48.983196 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:48.983251 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:48.983297 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:48.983327 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:48.983361 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:48.983399 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:48.983427 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:49.008088 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:49.010513 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:49.010586 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:49.047703 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:50.590875 8834 executor.go:111] Tasks: 0 done / 94 total; 51 can run W0512 07:42:50.591213 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:50.591320 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:50.592945 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:50.595465 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:50.595536 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:50.595748 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0512 07:42:50.612467 8834 executor.go:111] Tasks: 51 done / 94 total; 19 can run I0512 07:42:50.618118 8834 executor.go:111] Tasks: 70 done / 94 total; 22 can run I0512 07:42:50.618478 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:50.618674 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:50.619619 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:50.621219 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:50.621334 8834 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0512 07:42:50.621652 8834 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0512 07:42:50.678015 8834 target.go:112] Terraform output is in /tmp/test3297182564/out --- PASS: TestManyAddons (1.72s) === RUN TestManyAddonsCCMIRSA I0512 07:42:50.700834 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:50.700938 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:50.701029 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:50.701094 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:50.701230 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:50.701261 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:50.701292 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:50.701329 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:50.701359 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:50.728807 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:50.732350 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:50.732449 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:50.770485 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:52.369093 8834 executor.go:111] Tasks: 0 done / 110 total; 60 can run W0512 07:42:52.369887 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:52.369982 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:52.370221 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:52.370300 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:52.370330 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:52.371083 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0512 07:42:52.393327 8834 executor.go:111] Tasks: 60 done / 110 total; 26 can run I0512 07:42:52.400590 8834 executor.go:111] Tasks: 86 done / 110 total; 22 can run I0512 07:42:52.400953 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:52.401389 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:52.402619 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:52.403615 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:52.403766 8834 executor.go:111] Tasks: 108 done / 110 total; 2 can run I0512 07:42:52.404094 8834 executor.go:111] Tasks: 110 done / 110 total; 0 can run I0512 07:42:52.476972 8834 target.go:112] Terraform output is in /tmp/test528991987/out --- PASS: TestManyAddonsCCMIRSA (1.80s) === RUN TestCCM I0512 07:42:52.497381 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:52.497452 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:52.497506 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:52.497562 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:52.497664 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:52.497694 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:52.497725 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:52.497763 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:52.497794 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:52.527005 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:52.530440 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:52.530536 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:52.568502 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:54.116945 8834 executor.go:111] Tasks: 0 done / 95 total; 52 can run W0512 07:42:54.117248 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:54.117302 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:54.117287 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:54.117714 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:54.117749 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:54.117881 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0512 07:42:54.131425 8834 executor.go:111] Tasks: 52 done / 95 total; 19 can run I0512 07:42:54.137309 8834 executor.go:111] Tasks: 71 done / 95 total; 22 can run I0512 07:42:54.137716 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:54.138159 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:54.139361 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:54.140477 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:54.140663 8834 executor.go:111] Tasks: 93 done / 95 total; 2 can run I0512 07:42:54.141211 8834 executor.go:111] Tasks: 95 done / 95 total; 0 can run I0512 07:42:54.206582 8834 target.go:112] Terraform output is in /tmp/test3227916407/out --- PASS: TestCCM (1.73s) === RUN TestExternalDNS I0512 07:42:54.225657 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:54.225738 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:54.225824 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:54.225916 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:54.226029 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:54.226090 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:54.226148 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:54.226207 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:54.226255 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:54.255198 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:54.258359 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:54.258457 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:54.297675 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:54.406009 8834 executor.go:111] Tasks: 0 done / 86 total; 44 can run W0512 07:42:54.406505 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:54.406632 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0512 07:42:54.406672 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:54.406666 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:54.406750 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0512 07:42:54.406805 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:54.409295 8834 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0512 07:42:54.414437 8834 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0512 07:42:54.414782 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:54.415070 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:54.418987 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:54.420899 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:54.421037 8834 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0512 07:42:54.421846 8834 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0512 07:42:54.471778 8834 target.go:112] Terraform output is in /tmp/test3953488716/out I0512 07:42:54.476584 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:54.476661 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:54.477101 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:54.477234 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:54.477314 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:54.477385 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:54.477449 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:54.477529 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:54.477606 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:54.504343 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:54.507158 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:54.507258 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:54.544697 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0512 07:42:54.652127 8834 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0512 07:42:54.652400 8834 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0512 07:42:54.652434 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:54.652578 8834 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0512 07:42:54.652653 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0512 07:42:54.653188 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:54.653163 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:54.653349 8834 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0512 07:42:54.653382 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:54.653491 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0512 07:42:54.655090 8834 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0512 07:42:54.664514 8834 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0512 07:42:54.664797 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:54.665084 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:54.666252 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:54.667442 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:54.667588 8834 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0512 07:42:54.667930 8834 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0512 07:42:54.671625 8834 target.go:168] Cloudformation output is in /tmp/test2346556848/out --- PASS: TestExternalDNS (0.46s) === RUN TestExternalDNSIRSA I0512 07:42:54.681928 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:54.681979 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:54.682025 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:54.682074 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:54.682116 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:54.682144 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:54.682176 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:54.682213 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:54.682241 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:54.707028 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:54.709931 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:54.710019 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:54.748249 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:54.875801 8834 executor.go:111] Tasks: 0 done / 91 total; 47 can run I0512 07:42:54.876581 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:54.876621 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:54.876770 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0512 07:42:54.876721 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:54.876869 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:54.876934 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:54.879770 8834 executor.go:111] Tasks: 47 done / 91 total; 21 can run I0512 07:42:54.887268 8834 executor.go:111] Tasks: 68 done / 91 total; 21 can run I0512 07:42:54.887508 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:54.887849 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:54.888780 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:54.890235 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:54.890356 8834 executor.go:111] Tasks: 89 done / 91 total; 2 can run I0512 07:42:54.890734 8834 executor.go:111] Tasks: 91 done / 91 total; 0 can run I0512 07:42:54.950017 8834 target.go:112] Terraform output is in /tmp/test1023179002/out --- PASS: TestExternalDNSIRSA (0.27s) === RUN TestSharedSubnet I0512 07:42:54.955297 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:54.955366 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:54.955440 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:54.955529 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:54.955591 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:54.955648 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:54.955708 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:54.955769 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:54.955817 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:54.979948 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:54.982916 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:54.983031 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:55.021215 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:55.125027 8834 executor.go:111] Tasks: 0 done / 79 total; 43 can run W0512 07:42:55.125269 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:55.125768 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:55.125774 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:55.128427 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:55.125819 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:55.128960 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0512 07:42:55.132966 8834 executor.go:111] Tasks: 43 done / 79 total; 16 can run I0512 07:42:55.133376 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0512 07:42:55.138338 8834 executor.go:111] Tasks: 59 done / 79 total; 18 can run I0512 07:42:55.138779 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:55.138992 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:55.140237 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:55.141780 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:55.141896 8834 executor.go:111] Tasks: 77 done / 79 total; 2 can run I0512 07:42:55.142189 8834 executor.go:111] Tasks: 79 done / 79 total; 0 can run I0512 07:42:55.189872 8834 target.go:112] Terraform output is in /tmp/test2699358691/out --- PASS: TestSharedSubnet (0.24s) === RUN TestSharedVPC I0512 07:42:55.194440 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:55.194518 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:55.194575 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:55.194629 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:55.194672 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:55.194703 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:55.194735 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:55.194772 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:55.194799 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:55.219536 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:55.222566 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:55.222675 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:55.261214 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:55.366768 8834 executor.go:111] Tasks: 0 done / 83 total; 43 can run I0512 07:42:55.367243 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:55.367388 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:55.367631 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0512 07:42:55.367671 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:55.367663 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:55.367757 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0512 07:42:55.374820 8834 executor.go:111] Tasks: 43 done / 83 total; 17 can run I0512 07:42:55.375256 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0512 07:42:55.379994 8834 executor.go:111] Tasks: 60 done / 83 total; 21 can run I0512 07:42:55.380332 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:55.380692 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:55.381770 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:55.383112 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:55.383228 8834 executor.go:111] Tasks: 81 done / 83 total; 2 can run I0512 07:42:55.383604 8834 executor.go:111] Tasks: 83 done / 83 total; 0 can run I0512 07:42:55.433487 8834 target.go:112] Terraform output is in /tmp/test1340506779/out --- PASS: TestSharedVPC (0.24s) === RUN TestExistingIAM I0512 07:42:55.438121 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:55.438181 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:55.438234 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:55.438286 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:55.438328 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:55.438357 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:55.438389 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:55.438430 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:55.438461 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:55.470855 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:55.473869 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:55.473961 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:55.474021 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:55.474078 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:55.510968 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:55.615584 8834 executor.go:111] Tasks: 0 done / 94 total; 48 can run W0512 07:42:55.616272 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:55.616347 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:55.616445 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:55.616587 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:55.616644 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existing-iam.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:55.616695 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:55.616623 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:55.616756 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0512 07:42:55.623259 8834 executor.go:111] Tasks: 48 done / 94 total; 15 can run I0512 07:42:55.632508 8834 executor.go:111] Tasks: 63 done / 94 total; 27 can run I0512 07:42:55.632870 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:55.633193 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:55.633461 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:55.633736 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:55.635488 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:55.639489 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:55.639836 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:55.640247 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:55.640388 8834 executor.go:111] Tasks: 90 done / 94 total; 4 can run I0512 07:42:55.640833 8834 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0512 07:42:55.712457 8834 target.go:112] Terraform output is in /tmp/test683420114/out --- PASS: TestExistingIAM (0.28s) === RUN TestPhaseNetwork I0512 07:42:55.717210 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:55.717269 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:55.717319 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:55.717373 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:55.717417 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:55.717445 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:55.717482 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:55.717519 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:55.717548 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:55.750627 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:55.753939 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:55.754122 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:55.754191 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:55.793109 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:55.901018 8834 executor.go:111] Tasks: 0 done / 117 total; 47 can run I0512 07:42:55.901360 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.lifecyclephases.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:55.901484 8834 executor.go:111] Tasks: 47 done / 117 total; 28 can run I0512 07:42:55.901950 8834 executor.go:111] Tasks: 75 done / 117 total; 36 can run I0512 07:42:55.902360 8834 executor.go:111] Tasks: 111 done / 117 total; 5 can run I0512 07:42:55.902537 8834 executor.go:111] Tasks: 116 done / 117 total; 1 can run I0512 07:42:55.902667 8834 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0512 07:42:55.913877 8834 target.go:112] Terraform output is in /tmp/test2798383470/out --- PASS: TestPhaseNetwork (0.20s) === RUN TestExternalLoadBalancer I0512 07:42:55.916894 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:55.916969 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:55.917035 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:55.917085 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:55.917172 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:55.917205 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:55.917237 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:55.917271 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:55.917298 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:55.943292 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:55.946199 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:55.946307 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:55.984863 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:56.091942 8834 executor.go:111] Tasks: 0 done / 93 total; 51 can run W0512 07:42:56.092360 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:56.092556 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:56.092876 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0512 07:42:56.092992 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:56.092928 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:56.093052 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0512 07:42:56.095764 8834 executor.go:111] Tasks: 51 done / 93 total; 19 can run I0512 07:42:56.105824 8834 executor.go:111] Tasks: 70 done / 93 total; 21 can run I0512 07:42:56.106212 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.106356 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.107926 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.109010 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.109131 8834 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0512 07:42:56.109457 8834 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0512 07:42:56.164370 8834 target.go:112] Terraform output is in /tmp/test2377183497/out I0512 07:42:56.168648 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:56.168705 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:56.168755 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:56.168807 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:56.168851 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:56.168881 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:56.168914 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:56.168952 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:56.168983 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:56.194164 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:56.196985 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.197088 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.234553 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0512 07:42:56.340296 8834 executor.go:111] Tasks: 0 done / 93 total; 51 can run I0512 07:42:56.340893 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:56.341018 8834 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key W0512 07:42:56.341027 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:56.341035 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:56.341129 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } W0512 07:42:56.341326 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:56.341526 8834 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0512 07:42:56.341573 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:56.342849 8834 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0512 07:42:56.348910 8834 executor.go:111] Tasks: 51 done / 93 total; 19 can run I0512 07:42:56.354149 8834 executor.go:111] Tasks: 70 done / 93 total; 21 can run I0512 07:42:56.354604 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.354823 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.356089 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.357082 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.357228 8834 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0512 07:42:56.357538 8834 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0512 07:42:56.360915 8834 target.go:168] Cloudformation output is in /tmp/test2147736321/out --- PASS: TestExternalLoadBalancer (0.45s) === RUN TestPhaseIAM integration_test.go:633: unable to test w/o allowing failed validation --- SKIP: TestPhaseIAM (0.00s) === RUN TestPhaseCluster integration_test.go:641: unable to test w/o allowing failed validation --- SKIP: TestPhaseCluster (0.00s) === RUN TestMixedInstancesASG I0512 07:42:56.371881 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:56.371929 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:56.371971 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:56.372023 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:56.372103 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:56.372133 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:56.372163 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:56.372195 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:56.372223 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:56.410129 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:56.413323 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.413425 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.413504 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.413582 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.413652 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.413715 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.413794 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.451736 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:56.565075 8834 executor.go:111] Tasks: 0 done / 104 total; 50 can run W0512 07:42:56.565514 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:56.566019 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:56.566060 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:56.566197 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0512 07:42:56.566219 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:56.566239 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:56.566375 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:56.566416 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:56.567994 8834 executor.go:111] Tasks: 50 done / 104 total; 23 can run I0512 07:42:56.575782 8834 executor.go:111] Tasks: 73 done / 104 total; 27 can run I0512 07:42:56.576770 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.577006 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.577130 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.577316 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.578164 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.579153 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.579472 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.579628 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.579751 8834 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0512 07:42:56.580172 8834 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0512 07:42:56.663140 8834 target.go:112] Terraform output is in /tmp/test3074548218/out I0512 07:42:56.668142 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:56.668208 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:56.668263 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:56.668320 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:56.668366 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:56.668396 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:56.668427 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:56.668463 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:56.668495 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:56.701730 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:56.704666 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.704751 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.704806 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.704859 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.704904 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.704950 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.704995 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.743930 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0512 07:42:56.848801 8834 executor.go:111] Tasks: 0 done / 104 total; 50 can run W0512 07:42:56.849340 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:56.849444 8834 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0512 07:42:56.849469 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:56.849554 8834 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0512 07:42:56.849499 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:56.849632 8834 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0512 07:42:56.849647 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:56.849766 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:56.849722 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "mixedinstances.example.com" },{ Key: "KubernetesCluster", Value: "mixedinstances.example.com" },{ Key: "kubernetes.io/cluster/mixedinstances.example.com", Value: "owned" }] }] } W0512 07:42:56.849790 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0512 07:42:56.849769 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:56.857281 8834 executor.go:111] Tasks: 50 done / 104 total; 23 can run I0512 07:42:56.863873 8834 executor.go:111] Tasks: 73 done / 104 total; 27 can run I0512 07:42:56.864394 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.864615 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.864873 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.865042 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.868268 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.871271 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.871356 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.871990 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.872150 8834 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0512 07:42:56.872677 8834 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0512 07:42:56.877717 8834 target.go:168] Cloudformation output is in /tmp/test186534200/out --- PASS: TestMixedInstancesASG (0.52s) === RUN TestMixedInstancesSpotASG I0512 07:42:56.894334 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:56.894388 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:56.894436 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:56.894509 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:56.894597 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:56.894627 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:56.894659 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:56.894694 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:56.894723 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:56.925933 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:56.928341 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.928409 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.928451 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.928493 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.928534 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.928570 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.928608 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:56.966528 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:57.071634 8834 executor.go:111] Tasks: 0 done / 104 total; 50 can run W0512 07:42:57.071957 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:57.072174 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:57.073838 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:57.072229 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:57.074083 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:57.074265 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0512 07:42:57.074372 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:57.075693 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:57.080261 8834 executor.go:111] Tasks: 50 done / 104 total; 23 can run I0512 07:42:57.085586 8834 executor.go:111] Tasks: 73 done / 104 total; 27 can run I0512 07:42:57.085888 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:57.086110 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:57.086302 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:57.087653 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:57.088732 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:57.093099 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:57.093241 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:57.093360 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:57.093524 8834 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0512 07:42:57.094059 8834 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0512 07:42:57.170568 8834 target.go:112] Terraform output is in /tmp/test974082870/out I0512 07:42:57.179205 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:57.179295 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:57.179373 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:57.179453 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:57.179519 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:57.179570 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:57.179622 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:57.179677 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:57.179720 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:57.216358 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:57.219500 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:57.219588 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:57.219644 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:57.219700 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:57.219749 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:57.219798 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:57.219851 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:57.258513 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0512 07:42:57.367175 8834 executor.go:111] Tasks: 0 done / 104 total; 50 can run I0512 07:42:57.367731 8834 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0512 07:42:57.367771 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:57.367935 8834 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0512 07:42:57.367948 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0512 07:42:57.368034 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0512 07:42:57.368087 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:57.368053 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:57.368131 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0512 07:42:57.368234 8834 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0512 07:42:57.368265 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:57.368408 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/mixedinstances.example.com", Value: "owned" },{ Key: "Name", Value: "mixedinstances.example.com" },{ Key: "KubernetesCluster", Value: "mixedinstances.example.com" }] }] } I0512 07:42:57.371836 8834 executor.go:111] Tasks: 50 done / 104 total; 23 can run I0512 07:42:57.379812 8834 executor.go:111] Tasks: 73 done / 104 total; 27 can run I0512 07:42:57.380072 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:57.380390 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:57.380649 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:57.380930 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:57.381572 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:57.383141 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:57.383330 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:57.383543 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:57.383715 8834 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0512 07:42:57.384075 8834 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0512 07:42:57.392022 8834 target.go:168] Cloudformation output is in /tmp/test2044963562/out --- PASS: TestMixedInstancesSpotASG (0.51s) === RUN TestContainerd I0512 07:42:57.408686 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:57.408737 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:57.408783 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:57.408835 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:57.408916 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:57.408946 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:57.408978 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:57.409013 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:57.409043 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:57.433067 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:57.435806 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:57.435891 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:57.476610 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0512 07:42:57.580018 8834 executor.go:111] Tasks: 0 done / 86 total; 44 can run W0512 07:42:57.581318 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0512 07:42:57.581233 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:57.583595 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:57.583873 8834 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0512 07:42:57.583917 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:57.584066 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/containerd.example.com", Value: "owned" },{ Key: "Name", Value: "containerd.example.com" },{ Key: "KubernetesCluster", Value: "containerd.example.com" }] }] } I0512 07:42:57.584598 8834 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0512 07:42:57.584642 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:57.584929 8834 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0512 07:42:57.588106 8834 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0512 07:42:57.593573 8834 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0512 07:42:57.594039 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:57.594270 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:57.595583 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:57.596585 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:57.596818 8834 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0512 07:42:57.597207 8834 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0512 07:42:57.600357 8834 target.go:168] Cloudformation output is in /tmp/test2511457049/out --- PASS: TestContainerd (0.21s) === RUN TestContainerdCustom I0512 07:42:57.614751 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:57.614802 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:57.614850 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:57.614900 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:57.614987 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:57.615017 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:57.615049 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:57.615083 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:57.615111 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:57.640482 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:57.643536 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:57.643629 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:57.681704 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0512 07:42:57.787009 8834 executor.go:111] Tasks: 0 done / 86 total; 44 can run W0512 07:42:57.788575 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0512 07:42:57.789803 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:57.790502 8834 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0512 07:42:57.790508 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:57.790554 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:57.790712 8834 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0512 07:42:57.790776 8834 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0512 07:42:57.790806 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:57.791264 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "containerd.example.com" },{ Key: "KubernetesCluster", Value: "containerd.example.com" },{ Key: "kubernetes.io/cluster/containerd.example.com", Value: "owned" }] }] } I0512 07:42:57.794439 8834 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0512 07:42:57.800189 8834 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0512 07:42:57.800406 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:57.800680 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:57.802021 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:57.803504 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:57.803653 8834 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0512 07:42:57.804001 8834 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0512 07:42:57.806923 8834 target.go:168] Cloudformation output is in /tmp/test3440496986/out --- PASS: TestContainerdCustom (0.20s) === RUN TestDockerCustom I0512 07:42:57.816859 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:57.816912 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:57.816952 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:57.817002 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:57.817087 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:57.817117 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:57.817146 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:57.817180 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:57.817208 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:57.846929 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:42:57.850021 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:57.850123 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:57.888782 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0512 07:42:57.996822 8834 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0512 07:42:57.998283 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:57.998538 8834 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0512 07:42:57.998538 8834 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0512 07:42:57.998638 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:57.998574 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:57.998591 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:57.998745 8834 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0512 07:42:57.998732 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "docker.example.com" },{ Key: "kubernetes.io/cluster/docker.example.com", Value: "owned" },{ Key: "Name", Value: "docker.example.com" }] }] } W0512 07:42:57.999488 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:58.003962 8834 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0512 07:42:58.009206 8834 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0512 07:42:58.009607 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:58.009871 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:58.011164 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:58.012823 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:42:58.012963 8834 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0512 07:42:58.013247 8834 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0512 07:42:58.016494 8834 target.go:168] Cloudformation output is in /tmp/test2436794251/out --- PASS: TestDockerCustom (0.21s) === RUN TestAPIServerNodes I0512 07:42:58.027296 8834 featureflag.go:162] FeatureFlag "APIServerNodes"=true I0512 07:42:58.027703 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:58.027770 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:58.027984 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:58.028079 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:58.028175 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:58.028247 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:58.028311 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:58.028391 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:58.028466 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:58.058123 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.4 ********************************************************************************* I0512 07:42:58.061029 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:58.061126 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:58.061214 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:58.102771 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0512 07:42:58.269052 8834 executor.go:111] Tasks: 0 done / 98 total; 48 can run I0512 07:42:58.269364 8834 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0512 07:42:58.269390 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:58.269485 8834 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0512 07:42:58.269752 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:58.269895 8834 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key W0512 07:42:58.269903 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:58.269910 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:58.270010 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:58.270030 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } W0512 07:42:58.270146 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:42:58.272737 8834 executor.go:111] Tasks: 48 done / 98 total; 24 can run I0512 07:42:58.277911 8834 executor.go:111] Tasks: 72 done / 98 total; 23 can run I0512 07:42:58.278128 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:58.278558 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:58.278863 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:58.281382 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:58.282738 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:58.285121 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:58.285267 8834 executor.go:111] Tasks: 95 done / 98 total; 3 can run I0512 07:42:58.285610 8834 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0512 07:42:58.289379 8834 target.go:168] Cloudformation output is in /tmp/test2768675541/out I0512 07:42:58.303171 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:58.303224 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:58.303272 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:58.303324 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:58.303369 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:58.303398 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:58.303430 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:58.303464 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:58.303493 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:58.329179 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.4 ********************************************************************************* I0512 07:42:58.331672 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:58.331744 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:58.331799 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:58.368723 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:58.526711 8834 executor.go:111] Tasks: 0 done / 98 total; 48 can run W0512 07:42:58.526945 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:58.527129 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:58.527152 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:58.527220 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0512 07:42:58.527461 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:42:58.527691 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:58.527845 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:58.532988 8834 executor.go:111] Tasks: 48 done / 98 total; 24 can run I0512 07:42:58.539156 8834 executor.go:111] Tasks: 72 done / 98 total; 23 can run I0512 07:42:58.539392 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:58.539557 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:58.539853 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:58.540779 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:58.541013 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:58.542199 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:58.542336 8834 executor.go:111] Tasks: 95 done / 98 total; 3 can run I0512 07:42:58.542803 8834 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0512 07:42:58.609024 8834 target.go:112] Terraform output is in /tmp/test3666314537/out I0512 07:42:58.613729 8834 featureflag.go:162] FeatureFlag "APIServerNodes"=false --- PASS: TestAPIServerNodes (0.59s) === RUN TestNTHQueueProcessor I0512 07:42:58.614086 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:58.614146 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:58.614195 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:58.614250 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:58.614362 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:58.614395 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:58.614427 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:58.614484 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:58.614521 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:58.639491 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.13 ********************************************************************************* I0512 07:42:58.646118 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:58.646227 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:59.854243 8834 zones.go:93] ListHostedZonesPages { } I0512 07:42:59.982068 8834 executor.go:111] Tasks: 0 done / 98 total; 46 can run I0512 07:42:59.982417 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:42:59.982799 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:42:59.982820 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:42:59.982883 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0512 07:42:59.982952 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:42:59.983121 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:42:59.988797 8834 executor.go:111] Tasks: 46 done / 98 total; 23 can run I0512 07:42:59.994043 8834 executor.go:111] Tasks: 69 done / 98 total; 25 can run I0512 07:42:59.994528 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:59.994747 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:59.995627 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:59.997527 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:42:59.997650 8834 executor.go:111] Tasks: 94 done / 98 total; 2 can run I0512 07:42:59.998029 8834 executor.go:111] Tasks: 96 done / 98 total; 2 can run I0512 07:42:59.998259 8834 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0512 07:43:00.062160 8834 target.go:112] Terraform output is in /tmp/test2319267011/out I0512 07:43:00.067093 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:43:00.067155 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:43:00.067208 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:43:00.067264 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:43:00.067306 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:43:00.067334 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:43:00.067365 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:43:00.067399 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:43:00.067427 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:43:00.091850 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.13 ********************************************************************************* I0512 07:43:00.094876 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:00.094975 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:00.134204 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0512 07:43:00.255490 8834 executor.go:111] Tasks: 0 done / 98 total; 46 can run W0512 07:43:00.256138 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0512 07:43:00.256177 8834 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0512 07:43:00.256117 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0512 07:43:00.256349 8834 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0512 07:43:00.256376 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:00.257073 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" }] }] } I0512 07:43:00.258604 8834 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0512 07:43:00.258639 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:00.258765 8834 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0512 07:43:00.263690 8834 executor.go:111] Tasks: 46 done / 98 total; 23 can run I0512 07:43:00.269605 8834 executor.go:111] Tasks: 69 done / 98 total; 25 can run I0512 07:43:00.270022 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:00.270316 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:00.271473 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:00.272527 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:00.272671 8834 executor.go:111] Tasks: 94 done / 98 total; 2 can run I0512 07:43:00.273010 8834 executor.go:111] Tasks: 96 done / 98 total; 2 can run I0512 07:43:00.273387 8834 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0512 07:43:00.277116 8834 target.go:168] Cloudformation output is in /tmp/test2323294329/out --- PASS: TestNTHQueueProcessor (1.68s) === RUN TestCustomIRSA I0512 07:43:00.291634 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:43:00.291686 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:43:00.291737 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:43:00.291786 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:43:00.291879 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:43:00.291910 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:43:00.291943 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:43:00.291978 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:43:00.292006 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:43:00.321272 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.13 ********************************************************************************* I0512 07:43:00.324400 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:00.324510 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:00.361623 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:01.690205 8834 executor.go:111] Tasks: 0 done / 99 total; 51 can run W0512 07:43:01.690710 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:43:01.690847 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:43:01.690886 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:01.691000 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0512 07:43:01.691351 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:43:01.691405 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:01.706029 8834 executor.go:111] Tasks: 51 done / 99 total; 25 can run I0512 07:43:01.711864 8834 executor.go:111] Tasks: 76 done / 99 total; 21 can run I0512 07:43:01.712599 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:01.712826 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:01.714625 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:01.718455 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:01.718596 8834 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0512 07:43:01.718920 8834 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0512 07:43:01.782841 8834 target.go:112] Terraform output is in /tmp/test3464580058/out --- PASS: TestCustomIRSA (1.52s) === RUN TestCustomIRSA119 I0512 07:43:01.808585 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:43:01.808656 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:43:01.808707 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:43:01.808762 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:43:01.808809 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:43:01.808838 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:43:01.808871 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:43:01.808910 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:43:01.808940 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:43:01.836122 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.19.16 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.19.16 ********************************************************************************* I0512 07:43:01.839064 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:01.839143 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:01.878962 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:01.980112 8834 executor.go:111] Tasks: 0 done / 97 total; 49 can run W0512 07:43:01.980399 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:43:01.980512 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0512 07:43:01.980569 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:43:01.980534 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:01.980612 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0512 07:43:01.980739 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:01.982828 8834 executor.go:111] Tasks: 49 done / 97 total; 25 can run I0512 07:43:01.987998 8834 executor.go:111] Tasks: 74 done / 97 total; 21 can run I0512 07:43:01.988383 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:01.988620 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:01.989740 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:01.990756 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:01.990873 8834 executor.go:111] Tasks: 95 done / 97 total; 2 can run I0512 07:43:01.991243 8834 executor.go:111] Tasks: 97 done / 97 total; 0 can run I0512 07:43:02.052271 8834 target.go:112] Terraform output is in /tmp/test1983104748/out --- PASS: TestCustomIRSA119 (0.25s) === RUN TestClusterNameDigit I0512 07:43:02.060201 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:43:02.060268 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:43:02.060319 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:43:02.060374 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:43:02.060417 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:43:02.060446 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:43:02.060477 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:43:02.060614 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:43:02.060643 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:43:02.087943 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.13 ********************************************************************************* I0512 07:43:02.091001 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:02.091098 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:02.130073 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:02.235870 8834 executor.go:111] Tasks: 0 done / 94 total; 48 can run W0512 07:43:02.236343 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target W0512 07:43:02.236366 8834 warmpool.go:124] ASG warm pool is not supported by the terraform target I0512 07:43:02.236673 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.123.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:02.236807 8834 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0512 07:43:02.236841 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:02.236951 8834 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0512 07:43:02.239180 8834 executor.go:111] Tasks: 48 done / 94 total; 23 can run I0512 07:43:02.249445 8834 executor.go:111] Tasks: 71 done / 94 total; 21 can run I0512 07:43:02.249694 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:02.249957 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:02.250831 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:02.252434 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:02.252626 8834 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0512 07:43:02.252918 8834 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0512 07:43:02.312272 8834 target.go:112] Terraform output is in /tmp/test1651534655/out --- PASS: TestClusterNameDigit (0.26s) === RUN TestLifecycleMinimalAWS I0512 07:43:02.317322 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:43:02.317382 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:43:02.317433 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:43:02.317489 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:43:02.317531 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:43:02.317561 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:43:02.317591 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:43:02.317625 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:43:02.317656 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:43:02.317721 8834 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal.example.com I0512 07:43:02.340969 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:02.343835 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:02.343927 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0512 07:43:02.383335 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:02.487703 8834 iamrole.go:91] ListRoles: { } I0512 07:43:02.489274 8834 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0512 07:43:02.489640 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:02.489745 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:02.489896 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:02.489941 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I0512 07:43:02.490048 8834 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0512 07:43:02.490179 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0512 07:43:02.490235 8834 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.minimal.example.com", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] } I0512 07:43:02.492256 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0512 07:43:02.492502 8834 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0512 07:43:02.492514 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } I0512 07:43:02.492703 8834 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] } I0512 07:43:02.492636 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0512 07:43:02.492907 8834 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.minimal.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0512 07:43:02.493120 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0512 07:43:02.493387 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] }] } I0512 07:43:02.493667 8834 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0512 07:43:02.493905 8834 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0512 07:43:02.493991 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0512 07:43:02.494102 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:02.494316 8834 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0512 07:43:02.494788 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0512 07:43:02.494852 8834 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0512 07:43:02.494983 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0512 07:43:02.495911 8834 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0512 07:43:02.496095 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0512 07:43:02.497633 8834 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0512 07:43:02.497869 8834 iamrole.go:163] ListAttachedRolePolicies: masters.minimal.example.com I0512 07:43:02.497835 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0512 07:43:02.498042 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:02.498109 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.minimal.example.com I0512 07:43:02.498202 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0512 07:43:02.498374 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0512 07:43:02.498247 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0512 07:43:02.498454 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:02.498452 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0512 07:43:02.498521 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0512 07:43:02.498807 8834 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } I0512 07:43:02.498839 8834 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0512 07:43:02.498942 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0512 07:43:02.499023 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0512 07:43:02.499174 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0512 07:43:02.499255 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:02.499369 8834 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0512 07:43:02.499489 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0512 07:43:02.499796 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0512 07:43:02.499793 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0512 07:43:02.500012 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:43:02.500096 8834 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0512 07:43:02.500187 8834 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0512 07:43:02.500270 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0512 07:43:02.500343 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0512 07:43:02.500484 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0512 07:43:02.500554 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:02.500608 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:43:02.500660 8834 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0512 07:43:02.500793 8834 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0512 07:43:02.502990 8834 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0512 07:43:02.503309 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0512 07:43:02.503527 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:02.503785 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:02.503990 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0512 07:43:02.504179 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" }] }] } I0512 07:43:02.504433 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0512 07:43:02.504634 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-udp-1to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0512 07:43:02.504857 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:02.505001 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0512 07:43:02.505071 8834 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0512 07:43:02.505154 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-4003to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0512 07:43:02.505408 8834 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0512 07:43:02.505480 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-1to2379-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0512 07:43:02.505651 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0512 07:43:02.505971 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0512 07:43:02.506183 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0512 07:43:02.506248 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0512 07:43:02.506423 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0512 07:43:02.506696 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0512 07:43:02.506865 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-2382to4000-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0512 07:43:02.507031 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0512 07:43:02.507192 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:02.507318 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:02.507386 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:02.507556 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:02.510784 8834 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0512 07:43:02.511080 8834 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0512 07:43:02.511177 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:02.511219 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:02.511290 8834 dns.go:238] Pre-creating DNS records I0512 07:43:02.511324 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:02.516795 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:02.519511 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:02.519601 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0512 07:43:02.556620 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:02.663068 8834 iamrole.go:91] ListRoles: { } I0512 07:43:02.664684 8834 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0512 07:43:02.665181 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:02.665228 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:02.665298 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:02.665405 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0512 07:43:02.665672 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0512 07:43:02.665916 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0512 07:43:02.665980 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0512 07:43:02.666074 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:02.666140 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0512 07:43:02.666352 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0512 07:43:02.669000 8834 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0512 07:43:02.669566 8834 iamrole.go:163] ListAttachedRolePolicies: masters.minimal.example.com I0512 07:43:02.669463 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:02.669793 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.minimal.example.com I0512 07:43:02.669912 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0512 07:43:02.670933 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:02.673156 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0512 07:43:02.673429 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:02.673521 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0512 07:43:02.673733 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0512 07:43:02.673872 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0512 07:43:02.674054 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:02.674131 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:43:02.677162 8834 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0512 07:43:02.677350 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:02.677456 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:02.677686 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:02.677902 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:02.678108 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:02.678270 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:02.678481 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:02.680888 8834 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0512 07:43:02.681169 8834 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0512 07:43:02.684342 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0512 07:43:02.684408 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0512 07:43:02.684448 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0512 07:43:02.684515 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0512 07:43:02.684559 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0512 07:43:02.684595 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0512 07:43:02.684629 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0512 07:43:02.684664 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0512 07:43:02.684701 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0512 07:43:02.684735 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0512 07:43:02.684773 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0512 07:43:02.684816 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0512 07:43:02.684848 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0512 07:43:02.684880 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0512 07:43:02.684916 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0512 07:43:02.684949 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0512 07:43:02.684983 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0512 07:43:02.685016 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:43:02.685053 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0512 07:43:02.685085 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0512 07:43:02.685118 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0512 07:43:02.685150 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0512 07:43:02.685187 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0512 07:43:02.685224 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0512 07:43:02.685258 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0512 07:43:02.686747 8834 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0512 07:43:02.686783 8834 keypairs.go:95] DescribeKeyPairs: { } I0512 07:43:02.686857 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0512 07:43:02.686944 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0512 07:43:02.687042 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0512 07:43:02.687150 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0512 07:43:02.687242 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0512 07:43:02.687324 8834 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0512 07:43:02.687378 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0512 07:43:02.687420 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0512 07:43:02.687467 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0512 07:43:02.687520 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0512 07:43:02.687561 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } W0512 07:43:02.687608 8834 tags.go:48] unknown cluster tag on vpc:vpc-12345678: "kubernetes.io/cluster/minimal.example.com"="" W0512 07:43:02.687637 8834 api.go:53] PageSize not implemented I0512 07:43:02.687657 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0512 07:43:02.687688 8834 loadbalancers.go:34] PageSize not implemented I0512 07:43:02.687710 8834 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0512 07:43:02.687738 8834 targetgroups.go:35] PageSize not implemented I0512 07:43:02.687766 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0512 07:43:02.687842 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:02.687874 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:02.687923 8834 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0512 07:43:02.687963 8834 iamrole.go:91] ListRoles: { } I0512 07:43:02.688019 8834 internetgateways.go:101] DescribeInternetGateways: { } I0512 07:43:02.688065 8834 routetable.go:53] DescribeRouteTables: { } I0512 07:43:02.688098 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:master-us-test-1a.masters.minimal.example.com ok autoscaling-config:lt-1 ok autoscaling-config:lt-2 ok autoscaling-group:nodes.minimal.example.com ok I0512 07:43:02.688734 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:02.688808 8834 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0512 07:43:02.688813 8834 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0512 07:43:02.688921 8834 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } iam-instance-profile:masters.minimal.example.com ok I0512 07:43:02.688922 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:02.689038 8834 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } route53-record:Z1AFAKE1ZON3YO/A/api.minimal.example.com. ok I0512 07:43:02.689133 8834 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } iam-instance-profile:nodes.minimal.example.com ok I0512 07:43:02.688913 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0512 07:43:02.689189 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:02.689209 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0512 07:43:02.689264 8834 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0512 07:43:02.689326 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:02.689389 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0512 07:43:02.689617 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:02.689663 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0512 07:43:02.689717 8834 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0512 07:43:02.689824 8834 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0512 07:43:02.689933 8834 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0512 07:43:02.690140 8834 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal.example.com" } I0512 07:43:02.690174 8834 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0512 07:43:02.690229 8834 iamrole.go:182] ListAttachedRolePolicies: masters.minimal.example.com I0512 07:43:02.690276 8834 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0512 07:43:02.690321 8834 iamrole.go:139] DeleteRole: { RoleName: "masters.minimal.example.com" } iam-role:masters.minimal.example.com ok I0512 07:43:02.690357 8834 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal.example.com" } I0512 07:43:02.690403 8834 iamrole.go:182] ListAttachedRolePolicies: nodes.minimal.example.com I0512 07:43:02.690419 8834 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0512 07:43:02.690448 8834 iamrole.go:139] DeleteRole: { RoleName: "nodes.minimal.example.com" } iam-role:nodes.minimal.example.com ok I0512 07:43:02.690527 8834 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0512 07:43:02.690618 8834 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0512 07:43:02.690705 8834 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0512 07:43:02.690997 8834 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleMinimalAWS (0.37s) === RUN TestLifecycleMinimalOpenstack lifecycle_integration_test.go:433: running lifecycle test for cluster minimal-openstack.k8s.local I0512 07:43:02.700850 8834 subnets.go:182] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:02.769455 8834 apply_cluster.go:483] Gossip DNS: skipping DNS validation I0512 07:43:02.934568 8834 executor.go:111] Tasks: 0 done / 97 total; 43 can run I0512 07:43:02.941606 8834 executor.go:111] Tasks: 43 done / 97 total; 45 can run I0512 07:43:02.985364 8834 executor.go:111] Tasks: 88 done / 97 total; 5 can run I0512 07:43:02.989167 8834 executor.go:111] Tasks: 93 done / 97 total; 4 can run W0512 07:43:02.992686 8834 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0512 07:43:02.992896 8834 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0512 07:43:02.993041 8834 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. I0512 07:43:02.999230 8834 executor.go:111] Tasks: 97 done / 97 total; 0 can run ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:03.045239 8834 apply_cluster.go:483] Gossip DNS: skipping DNS validation I0512 07:43:03.206895 8834 executor.go:111] Tasks: 0 done / 97 total; 43 can run I0512 07:43:03.210616 8834 executor.go:111] Tasks: 43 done / 97 total; 45 can run I0512 07:43:03.222821 8834 executor.go:111] Tasks: 88 done / 97 total; 5 can run I0512 07:43:03.225383 8834 executor.go:111] Tasks: 93 done / 97 total; 4 can run W0512 07:43:03.228444 8834 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0512 07:43:03.228757 8834 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0512 07:43:03.229297 8834 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0512 07:43:03.229621 8834 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0512 07:43:03.230254 8834 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0512 07:43:03.230844 8834 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. I0512 07:43:03.233136 8834 executor.go:111] Tasks: 97 done / 97 total; 0 can run Instance:ba5d503b-7062-4365-a3e6-75123ef437b5 ok Port:175997fb-8b08-4c59-a474-8bbf02b0292e ok Network:8f741943-32b4-4d98-9acd-f9594af34b7b ok Instance:f3aea33b-30ac-4215-963e-41923759bcaf ok Subnet:6c314180-d0fc-4094-8585-9ab3c921262e ok Instance:2ff43adf-b7fe-440e-a5c1-e7efb37df07d ok SecurityGroup:13167ba8-924a-4e35-9fea-b09aebfa2f2e ok ServerGroup:ac733cbb-9037-4dbe-af36-1778535179af ok Port:af3c4a2a-2077-464d-98e8-4f03a97760bf ok ServerGroup:01c3e9a1-d8ff-40d8-a6b9-147249e04e5b ok Port:8671e116-6c20-4434-b894-a7e6269f7e20 ok SecurityGroup:a79bdbce-c728-4027-b3a0-4ab6ae696301 ok SecurityGroup:d25e883b-0f6e-45a3-9397-ccaea3110377 ok SSHKey:kubernetes-minimal-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok --- PASS: TestLifecycleMinimalOpenstack (1.66s) === RUN TestLifecycleMinimalGCE I0512 07:43:04.350234 8834 featureflag.go:162] FeatureFlag "AlphaAllowGCE"=true lifecycle_integration_test.go:488: running lifecycle test for cluster minimal-gce.example.com I0512 07:43:04.351906 8834 network.go:198] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* W0512 07:43:04.590406 8834 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0512 07:43:04.590500 8834 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0512 07:43:04.590664 8834 autoscalinggroup.go:118] enabling storage-rw for etcd backups W0512 07:43:04.590714 8834 autoscalinggroup.go:163] VMs will be configured to use the GCE default compute Service Account! This is an anti-pattern W0512 07:43:04.590736 8834 autoscalinggroup.go:164] Use a pre-created Service Account with the flag: --gce-service-account=account@projectname.iam.gserviceaccount.com I0512 07:43:04.590767 8834 autoscalinggroup.go:168] gsa: default I0512 07:43:04.590866 8834 autoscalinggroup.go:160] VMs using Service Account: default I0512 07:43:04.590896 8834 autoscalinggroup.go:168] gsa: default I0512 07:43:04.592680 8834 executor.go:111] Tasks: 0 done / 61 total; 38 can run I0512 07:43:04.596096 8834 executor.go:111] Tasks: 38 done / 61 total; 17 can run I0512 07:43:04.601219 8834 executor.go:111] Tasks: 55 done / 61 total; 4 can run I0512 07:43:04.608893 8834 executor.go:111] Tasks: 59 done / 61 total; 2 can run I0512 07:43:04.609168 8834 executor.go:111] Tasks: 61 done / 61 total; 0 can run I0512 07:43:04.609268 8834 dns.go:238] Pre-creating DNS records ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* W0512 07:43:04.820943 8834 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0512 07:43:04.821034 8834 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0512 07:43:04.821147 8834 autoscalinggroup.go:118] enabling storage-rw for etcd backups W0512 07:43:04.821196 8834 autoscalinggroup.go:163] VMs will be configured to use the GCE default compute Service Account! This is an anti-pattern W0512 07:43:04.821214 8834 autoscalinggroup.go:164] Use a pre-created Service Account with the flag: --gce-service-account=account@projectname.iam.gserviceaccount.com I0512 07:43:04.821229 8834 autoscalinggroup.go:168] gsa: default I0512 07:43:04.821287 8834 autoscalinggroup.go:160] VMs using Service Account: default I0512 07:43:04.821306 8834 autoscalinggroup.go:168] gsa: default I0512 07:43:04.822577 8834 executor.go:111] Tasks: 0 done / 61 total; 38 can run I0512 07:43:04.830135 8834 executor.go:111] Tasks: 38 done / 61 total; 17 can run I0512 07:43:04.835162 8834 executor.go:111] Tasks: 55 done / 61 total; 4 can run I0512 07:43:04.842373 8834 executor.go:111] Tasks: 59 done / 61 total; 2 can run I0512 07:43:04.842549 8834 executor.go:111] Tasks: 61 done / 61 total; 0 can run I0512 07:43:04.844202 8834 gce.go:93] Scanning zones: [us-test1-a] FirewallRule:nodeport-external-to-node-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-ipv6-minimal-gce-example-com ok FirewallRule:master-to-master-minimal-gce-example-com ok Disk:d1-etcd-events-minimal-gce-example-com ok FirewallRule:node-to-master-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-master-us-test1-a-minimal-gce-example-com ok FirewallRule:node-to-node-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-nodes-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-ipv6-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-ipv6-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-ipv6-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-minimal-gce-example-com ok FirewallRule:master-to-node-minimal-gce-example-com ok Disk:d1-etcd-main-minimal-gce-example-com ok InstanceTemplate:nodes-minimal-gce-example-com-1652341384 ok InstanceTemplate:master-us-test1-a-minimal-do16cp-1652341384 ok Subnet:us-test1-minimal-gce-example-com ok Network:minimal-gce-example-com ok --- PASS: TestLifecycleMinimalGCE (0.50s) === RUN TestLifecycleFloatingIPOpenstack lifecycle_integration_test.go:433: running lifecycle test for cluster floatingip-openstack.k8s.local I0512 07:43:04.854536 8834 subnets.go:182] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:04.925143 8834 apply_cluster.go:483] Gossip DNS: skipping DNS validation I0512 07:43:05.093570 8834 executor.go:111] Tasks: 0 done / 99 total; 44 can run I0512 07:43:05.102720 8834 executor.go:111] Tasks: 44 done / 99 total; 45 can run I0512 07:43:05.153403 8834 executor.go:111] Tasks: 89 done / 99 total; 6 can run I0512 07:43:05.157006 8834 executor.go:111] Tasks: 95 done / 99 total; 4 can run W0512 07:43:05.159865 8834 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0512 07:43:05.160122 8834 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0512 07:43:05.160233 8834 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. I0512 07:43:05.165276 8834 executor.go:111] Tasks: 99 done / 99 total; 0 can run ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:05.211000 8834 apply_cluster.go:483] Gossip DNS: skipping DNS validation I0512 07:43:05.363241 8834 executor.go:111] Tasks: 0 done / 99 total; 44 can run I0512 07:43:05.372521 8834 executor.go:111] Tasks: 44 done / 99 total; 45 can run I0512 07:43:05.384861 8834 executor.go:111] Tasks: 89 done / 99 total; 6 can run I0512 07:43:05.387052 8834 executor.go:111] Tasks: 95 done / 99 total; 4 can run W0512 07:43:05.389772 8834 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0512 07:43:05.390589 8834 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0512 07:43:05.390964 8834 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0512 07:43:05.391511 8834 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0512 07:43:05.391904 8834 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0512 07:43:05.391971 8834 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. I0512 07:43:05.393991 8834 executor.go:111] Tasks: 99 done / 99 total; 0 can run Router-IF:b390e097-ec9e-4489-8c53-d3739eb2b2fe ok Port:d5965008-8c78-4899-9645-20f715c38c95 ok Port:518fac94-c368-43f5-ae30-456aab3b076d ok Router:56653818-1011-4642-a347-ab270b295c39 ok SecurityGroup:3dd007d8-671e-42ca-b9d2-e7245f321e60 ok Port:e22977af-6fa8-46a0-8b8b-1e5c95955c60 ok SecurityGroup:095495cf-1cf8-4022-aad6-1c47ccad5cfe ok Network:2d1f05ea-1672-4563-bcff-768df2b0b7f6 ok Instance:de102342-7c33-4c66-ae89-a5c058fd84a0 ok SecurityGroup:a603e435-8c5c-44c9-a552-f86b4c99bbc7 ok ServerGroup:9d9cafd8-2da2-4f6c-8bb2-90c139311b1b ok ServerGroup:4e4e31c5-16e3-4758-af7c-4c623f88aa29 ok Instance:c506af3d-97e9-430e-9a3c-023a542d0045 ok SSHKey:kubernetes-floatingip-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok Port:a15fda5b-877b-4972-9d1c-173b3b9e6029 ok Subnet:b390e097-ec9e-4489-8c53-d3739eb2b2fe ok Instance:df03602e-5b28-489f-8a3f-564b56102842 ok --- PASS: TestLifecycleFloatingIPOpenstack (1.66s) === RUN TestLifecyclePrivateCalico I0512 07:43:06.510161 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:43:06.510217 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:43:06.510268 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:43:06.510319 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:43:06.510361 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:43:06.510389 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:43:06.510418 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:43:06.510450 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:43:06.510478 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:43:06.510541 8834 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatecalico.example.com I0512 07:43:06.537985 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:06.540949 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:06.541046 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:06.541110 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:06.578653 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:06.898301 8834 iamrole.go:91] ListRoles: { } I0512 07:43:06.900599 8834 executor.go:111] Tasks: 0 done / 119 total; 48 can run I0512 07:43:06.901033 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0512 07:43:06.901104 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:06.901267 8834 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0512 07:43:06.901273 8834 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "bastions.privatecalico.example.com", Tags: [{ Key: "Name", Value: "bastions.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0512 07:43:06.901278 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0512 07:43:06.901533 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } I0512 07:43:06.901616 8834 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] } I0512 07:43:06.901691 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } I0512 07:43:06.901611 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0512 07:43:06.901760 8834 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0512 07:43:06.901874 8834 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0512 07:43:06.902077 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0512 07:43:06.901858 8834 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.privatecalico.example.com", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] } I0512 07:43:06.902166 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } I0512 07:43:06.902145 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0512 07:43:06.902249 8834 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0512 07:43:06.902330 8834 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0512 07:43:06.902450 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0512 07:43:06.902398 8834 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.privatecalico.example.com", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0512 07:43:06.902499 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:06.902548 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:06.902638 8834 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "privatecalico.example.com" }] }] } I0512 07:43:06.902797 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0512 07:43:06.902854 8834 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0512 07:43:06.902892 8834 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0512 07:43:06.902919 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0512 07:43:06.902964 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:06.903011 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0512 07:43:06.903697 8834 executor.go:111] Tasks: 48 done / 119 total; 28 can run I0512 07:43:06.903820 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:06.903947 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0512 07:43:06.903997 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0512 07:43:06.904138 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.privatecalico.example.com I0512 07:43:06.904123 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0512 07:43:06.904224 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0512 07:43:06.904248 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0512 07:43:06.904446 8834 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0512 07:43:06.904443 8834 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0512 07:43:06.904563 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:06.904603 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0512 07:43:06.904728 8834 iamrole.go:163] ListAttachedRolePolicies: masters.privatecalico.example.com I0512 07:43:06.904768 8834 iamrole.go:163] ListAttachedRolePolicies: bastions.privatecalico.example.com I0512 07:43:06.904768 8834 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0512 07:43:06.904816 8834 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0512 07:43:06.904798 8834 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0512 07:43:06.904912 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0512 07:43:06.904968 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0512 07:43:06.905170 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0512 07:43:06.905415 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "api-elb.privatecalico.example.com" }] }], VpcId: "vpc-1" } I0512 07:43:06.905516 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] }] } I0512 07:43:06.905617 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["privatecalico.example.com"] }] } I0512 07:43:06.905705 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0512 07:43:06.905806 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I0512 07:43:06.905919 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0512 07:43:06.906004 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0512 07:43:06.905972 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0512 07:43:06.906087 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0512 07:43:06.906121 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0512 07:43:06.906171 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0512 07:43:06.906302 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0512 07:43:06.906379 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:06.906429 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0512 07:43:06.906570 8834 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I0512 07:43:06.906641 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0512 07:43:06.906822 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastion-elb.privatecalico.example.com" }] }], VpcId: "vpc-1" } I0512 07:43:06.906930 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0512 07:43:06.906825 8834 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0512 07:43:06.906980 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0512 07:43:06.907063 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:43:06.906693 8834 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0512 07:43:06.907121 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:43:06.907163 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0512 07:43:06.907200 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0512 07:43:06.907252 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0512 07:43:06.907293 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:06.907330 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0512 07:43:06.907381 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0512 07:43:06.907426 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0512 07:43:06.907473 8834 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" } ] } I0512 07:43:06.907566 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0512 07:43:06.907611 8834 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatecalico.example.com" } ] } I0512 07:43:06.907697 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0512 07:43:06.907744 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0512 07:43:06.908802 8834 executor.go:111] Tasks: 76 done / 119 total; 37 can run I0512 07:43:06.908988 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } W0512 07:43:06.909135 8834 api.go:53] PageSize not implemented I0512 07:43:06.909140 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0512 07:43:06.909235 8834 api.go:53] PageSize not implemented I0512 07:43:06.909245 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:06.909301 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0512 07:43:06.909420 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0512 07:43:06.909488 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0512 07:43:06.909449 8834 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"], Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatecalico.example.com" }] } I0512 07:43:06.909538 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0512 07:43:06.909575 8834 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0512 07:43:06.909634 8834 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0512 07:43:06.909545 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-4003to65535-masters.privatecalico.example.com" }] }] } I0512 07:43:06.909728 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0512 07:43:06.909797 8834 tags.go:74] AddTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"], Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "api.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] } I0512 07:43:06.909853 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0512 07:43:06.909883 8834 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0512 07:43:06.909929 8834 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0512 07:43:06.909856 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-2382to4000-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0512 07:43:06.910165 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0512 07:43:06.910323 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0512 07:43:06.910445 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-udp-1to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0512 07:43:06.910582 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:06.910671 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:06.910766 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:06.910858 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0512 07:43:06.910903 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0512 07:43:06.911499 8834 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0512 07:43:06.911005 8834 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0512 07:43:06.912251 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0512 07:43:06.912513 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } I0512 07:43:06.912657 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-masters.privatecalico.example.com" }] }] } I0512 07:43:06.912833 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:06.912968 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:06.913059 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0512 07:43:06.913422 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:06.913659 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:06.914063 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0512 07:43:06.914416 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0512 07:43:06.915693 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" }] }] } I0512 07:43:06.915905 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0512 07:43:06.915955 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:06.916057 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:06.916960 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0512 07:43:06.917160 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-::/0" }] }] } I0512 07:43:06.917287 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0512 07:43:06.917426 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-::/0" }] }] } I0512 07:43:06.917552 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-ingress-tcp-22to22-bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0512 07:43:06.917725 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-4-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0512 07:43:06.917887 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I0512 07:43:06.918015 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0512 07:43:06.918144 8834 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0512 07:43:06.918243 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" }] }] } I0512 07:43:06.918395 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-1to2379-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0512 07:43:06.918536 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0512 07:43:06.918799 8834 executor.go:111] Tasks: 113 done / 119 total; 5 can run I0512 07:43:06.919044 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0512 07:43:06.919094 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:06.919223 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0512 07:43:06.919245 8834 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:43:06.919421 8834 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0512 07:43:06.919524 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0512 07:43:06.919572 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:43:06.919614 8834 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0512 07:43:06.919685 8834 executor.go:111] Tasks: 118 done / 119 total; 1 can run I0512 07:43:06.919748 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:06.919870 8834 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I0512 07:43:06.919934 8834 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0512 07:43:06.919984 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:06.920016 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:06.920069 8834 dns.go:238] Pre-creating DNS records I0512 07:43:06.920093 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:06.926720 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:06.930065 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:06.930159 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:06.930226 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:06.967095 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:07.299366 8834 iamrole.go:91] ListRoles: { } I0512 07:43:07.301255 8834 executor.go:111] Tasks: 0 done / 119 total; 48 can run I0512 07:43:07.301672 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0512 07:43:07.301876 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:07.301989 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0512 07:43:07.304357 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0512 07:43:07.304429 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0512 07:43:07.304577 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:07.304649 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0512 07:43:07.304839 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0512 07:43:07.304911 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:07.304971 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:07.309231 8834 executor.go:111] Tasks: 48 done / 119 total; 28 can run I0512 07:43:07.309450 8834 iamrole.go:163] ListAttachedRolePolicies: bastions.privatecalico.example.com I0512 07:43:07.309390 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:07.309544 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0512 07:43:07.309671 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:07.309727 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.privatecalico.example.com I0512 07:43:07.309778 8834 iamrole.go:163] ListAttachedRolePolicies: masters.privatecalico.example.com I0512 07:43:07.309738 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0512 07:43:07.309877 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:07.309918 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0512 07:43:07.310021 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:07.310055 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I0512 07:43:07.310153 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0512 07:43:07.310187 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0512 07:43:07.310310 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0512 07:43:07.310376 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0512 07:43:07.310566 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0512 07:43:07.310737 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0512 07:43:07.310780 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0512 07:43:07.310904 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:43:07.310943 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0512 07:43:07.311089 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0512 07:43:07.311173 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0512 07:43:07.314453 8834 executor.go:111] Tasks: 76 done / 119 total; 37 can run I0512 07:43:07.314673 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0512 07:43:07.314970 8834 api.go:53] PageSize not implemented I0512 07:43:07.315013 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I0512 07:43:07.315147 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0512 07:43:07.315094 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:07.315209 8834 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatecalico-example-0uch4k" } W0512 07:43:07.315270 8834 api.go:53] PageSize not implemented I0512 07:43:07.315303 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I0512 07:43:07.315402 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0512 07:43:07.315458 8834 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0512 07:43:07.315381 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:07.315563 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:07.315682 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:07.315875 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:07.315984 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:07.316096 8834 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0512 07:43:07.316186 8834 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0512 07:43:07.316251 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0512 07:43:07.316363 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:07.316480 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:07.316736 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:07.316932 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:07.319425 8834 executor.go:111] Tasks: 113 done / 119 total; 5 can run I0512 07:43:07.319524 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0512 07:43:07.319610 8834 dnsname.go:121] AliasTarget for "api.privatecalico.example.com" is "api-privatecalico-example-0uch4k.elb.cloudmock.com" W0512 07:43:07.319635 8834 api.go:53] PageSize not implemented I0512 07:43:07.319673 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:07.319684 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I0512 07:43:07.319750 8834 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } W0512 07:43:07.319940 8834 api.go:53] PageSize not implemented I0512 07:43:07.319968 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I0512 07:43:07.320080 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0512 07:43:07.320210 8834 executor.go:111] Tasks: 118 done / 119 total; 1 can run I0512 07:43:07.320293 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:07.320405 8834 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0512 07:43:07.324014 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0512 07:43:07.324080 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0512 07:43:07.324120 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0512 07:43:07.324154 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0512 07:43:07.324193 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0512 07:43:07.324229 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0512 07:43:07.324264 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0512 07:43:07.324298 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0512 07:43:07.324332 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0512 07:43:07.324368 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0512 07:43:07.324403 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0512 07:43:07.324441 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0512 07:43:07.324500 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0512 07:43:07.324561 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0512 07:43:07.324599 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0512 07:43:07.324634 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0512 07:43:07.324669 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0512 07:43:07.324706 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0512 07:43:07.324740 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0512 07:43:07.324779 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0512 07:43:07.324816 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0512 07:43:07.324852 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0512 07:43:07.324888 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0512 07:43:07.324924 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0512 07:43:07.324960 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:43:07.324998 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0512 07:43:07.325032 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0512 07:43:07.325069 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0512 07:43:07.325102 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0512 07:43:07.325135 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0512 07:43:07.325175 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0512 07:43:07.325212 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0512 07:43:07.325248 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0512 07:43:07.326836 8834 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0512 07:43:07.326865 8834 keypairs.go:95] DescribeKeyPairs: { } I0512 07:43:07.326906 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0512 07:43:07.326971 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0512 07:43:07.327054 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0512 07:43:07.327116 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0512 07:43:07.327164 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0512 07:43:07.327217 8834 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0512 07:43:07.327262 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0512 07:43:07.327309 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0512 07:43:07.327370 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0512 07:43:07.327426 8834 routetable.go:53] DescribeRouteTables: { } I0512 07:43:07.327456 8834 natgateway.go:117] DescribeNatGateways: { } I0512 07:43:07.327494 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0512 07:43:07.327536 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } W0512 07:43:07.327586 8834 api.go:53] PageSize not implemented I0512 07:43:07.327603 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I0512 07:43:07.327657 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0512 07:43:07.327686 8834 loadbalancers.go:34] PageSize not implemented I0512 07:43:07.327705 8834 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0512 07:43:07.327730 8834 targetgroups.go:35] PageSize not implemented I0512 07:43:07.327749 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0512 07:43:07.327832 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:07.327865 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:07.327912 8834 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0512 07:43:07.327958 8834 iamrole.go:91] ListRoles: { } I0512 07:43:07.328006 8834 internetgateways.go:101] DescribeInternetGateways: { } I0512 07:43:07.328059 8834 routetable.go:53] DescribeRouteTables: { } I0512 07:43:07.328104 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0512 07:43:07.328143 8834 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0512 07:43:07.328180 8834 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0512 07:43:07.328835 8834 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatecalico-example-0uch4k" } I0512 07:43:07.328870 8834 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } autoscaling-group:nodes.privatecalico.example.com ok I0512 07:43:07.328913 8834 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } volume:vol-2 ok load-balancer:bastion-privatecalico-exa-hocohm ok load-balancer:api-privatecalico-example-0uch4k ok I0512 07:43:07.328965 8834 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-group:master-us-test-1a.masters.privatecalico.example.com ok I0512 07:43:07.329006 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0512 07:43:07.329068 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W0512 07:43:07.329167 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:07.329184 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok autoscaling-config:lt-1 ok autoscaling-config:lt-3 ok autoscaling-group:bastion.privatecalico.example.com ok I0512 07:43:07.329096 8834 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0512 07:43:07.329308 8834 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } iam-instance-profile:nodes.privatecalico.example.com ok I0512 07:43:07.329244 8834 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0512 07:43:07.329198 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.privatecalico.example.com. ok I0512 07:43:07.329391 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0512 07:43:07.329381 8834 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0512 07:43:07.329531 8834 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } iam-instance-profile:bastions.privatecalico.example.com ok I0512 07:43:07.329450 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } W0512 07:43:07.329578 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:07.329561 8834 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0512 07:43:07.329608 8834 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } iam-instance-profile:masters.privatecalico.example.com ok I0512 07:43:07.329606 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0512 07:43:07.329648 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:07.329706 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W0512 07:43:07.329925 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:07.329949 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0512 07:43:07.329995 8834 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0512 07:43:07.330058 8834 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0512 07:43:07.330115 8834 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-2 ok I0512 07:43:07.330220 8834 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0512 07:43:07.330244 8834 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatecalico.example.com" } I0512 07:43:07.330269 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0512 07:43:07.330330 8834 iamrole.go:182] ListAttachedRolePolicies: masters.privatecalico.example.com I0512 07:43:07.330362 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0512 07:43:07.330466 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:07.330430 8834 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0512 07:43:07.330507 8834 iamrole.go:139] DeleteRole: { RoleName: "masters.privatecalico.example.com" } iam-role:masters.privatecalico.example.com ok I0512 07:43:07.330492 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:07.330553 8834 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatecalico.example.com" } I0512 07:43:07.330610 8834 iamrole.go:182] ListAttachedRolePolicies: nodes.privatecalico.example.com I0512 07:43:07.330626 8834 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0512 07:43:07.330659 8834 iamrole.go:139] DeleteRole: { RoleName: "nodes.privatecalico.example.com" } iam-role:nodes.privatecalico.example.com ok I0512 07:43:07.330586 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } I0512 07:43:07.330693 8834 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatecalico.example.com" } W0512 07:43:07.330721 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:07.330737 8834 iamrole.go:182] ListAttachedRolePolicies: bastions.privatecalico.example.com I0512 07:43:07.330750 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0512 07:43:07.330750 8834 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0512 07:43:07.330807 8834 iamrole.go:139] DeleteRole: { RoleName: "bastions.privatecalico.example.com" } iam-role:bastions.privatecalico.example.com ok I0512 07:43:07.330794 8834 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0512 07:43:07.330886 8834 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I0512 07:43:07.330953 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0512 07:43:07.331035 8834 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0512 07:43:07.331101 8834 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0512 07:43:07.331208 8834 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0512 07:43:07.331295 8834 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0512 07:43:07.331570 8834 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateCalico (0.82s) === RUN TestLifecyclePrivateKopeio I0512 07:43:07.332089 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:43:07.332135 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:43:07.332173 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:43:07.332219 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:43:07.332262 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:43:07.332290 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:43:07.332319 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:43:07.332353 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:43:07.332381 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:43:07.332440 8834 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatekopeio.example.com I0512 07:43:07.365848 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:07.368918 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:07.369014 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:07.369076 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:07.409598 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:07.532460 8834 iamrole.go:91] ListRoles: { } I0512 07:43:07.534891 8834 executor.go:111] Tasks: 0 done / 124 total; 48 can run I0512 07:43:07.535181 8834 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "bastions.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "bastions.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0512 07:43:07.535514 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } I0512 07:43:07.535541 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:07.535686 8834 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "bastions.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0512 07:43:07.535685 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.535888 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.535942 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } I0512 07:43:07.538093 8834 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0512 07:43:07.538040 8834 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0512 07:43:07.538288 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } I0512 07:43:07.538405 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:07.538493 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.538477 8834 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.privatekopeio.example.com", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.privatekopeio.example.com" }] } I0512 07:43:07.538746 8834 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0512 07:43:07.538830 8834 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.privatekopeio.example.com" }] } I0512 07:43:07.538922 8834 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0512 07:43:07.538951 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.539019 8834 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0512 07:43:07.539154 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0512 07:43:07.539310 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:07.539614 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0512 07:43:07.539972 8834 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0512 07:43:07.540042 8834 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0512 07:43:07.540118 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0512 07:43:07.540211 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0512 07:43:07.540346 8834 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0512 07:43:07.540770 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0512 07:43:07.541041 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:07.541695 8834 executor.go:111] Tasks: 48 done / 124 total; 31 can run I0512 07:43:07.541927 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0512 07:43:07.541874 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0512 07:43:07.542779 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.542988 8834 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0512 07:43:07.543004 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0512 07:43:07.543089 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0512 07:43:07.543120 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0512 07:43:07.543199 8834 iamrole.go:163] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0512 07:43:07.543142 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.543240 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0512 07:43:07.543279 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0512 07:43:07.543312 8834 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0512 07:43:07.543417 8834 iamrole.go:163] ListAttachedRolePolicies: masters.privatekopeio.example.com I0512 07:43:07.543414 8834 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0512 07:43:07.543429 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:07.543649 8834 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0512 07:43:07.543632 8834 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0512 07:43:07.543729 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I0512 07:43:07.543984 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastion-elb.privatekopeio.example.com" }] }], VpcId: "vpc-1" } I0512 07:43:07.544096 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.544242 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.544343 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0512 07:43:07.544428 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.8.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" } ] }], VpcId: "vpc-1" } I0512 07:43:07.544497 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.544608 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.544697 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0512 07:43:07.544717 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.544809 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0512 07:43:07.544898 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:07.544947 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0512 07:43:07.545001 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0512 07:43:07.545120 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.545211 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0512 07:43:07.545315 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0512 07:43:07.545360 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0512 07:43:07.545296 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0512 07:43:07.545437 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:07.545515 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0512 07:43:07.545576 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0512 07:43:07.545481 8834 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I0512 07:43:07.545617 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0512 07:43:07.545706 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0512 07:43:07.545755 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0512 07:43:07.545797 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1b"] }] } I0512 07:43:07.546010 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0512 07:43:07.545989 8834 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" }, { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0512 07:43:07.546091 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0512 07:43:07.546172 8834 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0512 07:43:07.546240 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0512 07:43:07.546338 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0512 07:43:07.546390 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0512 07:43:07.545840 8834 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0512 07:43:07.546430 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0512 07:43:07.546864 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0512 07:43:07.546970 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0512 07:43:07.547021 8834 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" }, { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" } ] } I0512 07:43:07.547102 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0512 07:43:07.547144 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0512 07:43:07.547206 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0512 07:43:07.547245 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0512 07:43:07.547298 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0512 07:43:07.547341 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:07.547390 8834 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] } I0512 07:43:07.547474 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0512 07:43:07.547526 8834 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" } ] } I0512 07:43:07.547614 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:43:07.547715 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:43:07.549910 8834 executor.go:111] Tasks: 79 done / 124 total; 39 can run I0512 07:43:07.550049 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0512 07:43:07.550324 8834 api.go:53] PageSize not implemented I0512 07:43:07.550261 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0512 07:43:07.550437 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:07.550467 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0512 07:43:07.550566 8834 tags.go:74] AddTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "api.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] } I0512 07:43:07.550578 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:07.550639 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0512 07:43:07.550677 8834 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0512 07:43:07.550729 8834 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0512 07:43:07.550748 8834 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } W0512 07:43:07.550828 8834 api.go:53] PageSize not implemented I0512 07:43:07.550789 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0512 07:43:07.550866 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0512 07:43:07.550930 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0512 07:43:07.551063 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-4"] }] } I0512 07:43:07.551100 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0512 07:43:07.551107 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-4" } I0512 07:43:07.551285 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0512 07:43:07.551359 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0512 07:43:07.551171 8834 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"], Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatekopeio.example.com" }] } I0512 07:43:07.551481 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0512 07:43:07.551403 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0512 07:43:07.551552 8834 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0512 07:43:07.551616 8834 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0512 07:43:07.551588 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0512 07:43:07.551731 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0512 07:43:07.551863 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:07.551942 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:07.552025 8834 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0512 07:43:07.552127 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-ingress-tcp-22to22-bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0512 07:43:07.552298 8834 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0512 07:43:07.552404 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I0512 07:43:07.552578 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0512 07:43:07.552784 8834 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0512 07:43:07.552858 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0512 07:43:07.553084 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-4003to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0512 07:43:07.553273 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatekopeio.example.com" }] }] } I0512 07:43:07.553396 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:07.553481 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-2382to4000-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0512 07:43:07.553618 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I0512 07:43:07.553746 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0512 07:43:07.553792 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0512 07:43:07.553909 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0512 07:43:07.553950 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-udp-1to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0512 07:43:07.554087 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" }] }] } I0512 07:43:07.554220 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0512 07:43:07.554345 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0512 07:43:07.554482 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0512 07:43:07.554623 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0512 07:43:07.554672 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-1to2379-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0512 07:43:07.554817 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:07.554901 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0512 07:43:07.555035 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0512 07:43:07.555247 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } I0512 07:43:07.555406 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0512 07:43:07.555625 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0512 07:43:07.555697 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:07.555843 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:07.555910 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0512 07:43:07.555944 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-3" } I0512 07:43:07.555985 8834 tags.go:42] CreateTags { Resources: ["nat-a2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0512 07:43:07.556030 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:07.556074 8834 tags.go:42] CreateTags { Resources: ["nat-b2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0512 07:43:07.556122 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:43:07.556172 8834 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0512 07:43:07.556214 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0512 07:43:07.556256 8834 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0512 07:43:07.556348 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:07.556504 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0512 07:43:07.556591 8834 tags.go:42] CreateTags { Resources: ["subnet-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0512 07:43:07.556641 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0512 07:43:07.556684 8834 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0512 07:43:07.558208 8834 executor.go:111] Tasks: 118 done / 124 total; 6 can run I0512 07:43:07.558314 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0512 07:43:07.558339 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0512 07:43:07.558473 8834 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-b2345678", RouteTableId: "rtb-3" } I0512 07:43:07.558532 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:07.558508 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0512 07:43:07.558669 8834 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-a2345678", RouteTableId: "rtb-2" } I0512 07:43:07.558739 8834 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0512 07:43:07.558790 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:07.558822 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:07.558879 8834 dns.go:238] Pre-creating DNS records I0512 07:43:07.558904 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:07.565189 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:07.568252 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:07.568346 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:07.568413 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:07.605907 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:07.728662 8834 iamrole.go:91] ListRoles: { } I0512 07:43:07.731126 8834 executor.go:111] Tasks: 0 done / 124 total; 48 can run I0512 07:43:07.731648 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.731942 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.732052 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:07.732159 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:07.732313 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:07.732497 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.732626 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0512 07:43:07.732659 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0512 07:43:07.732710 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:07.732767 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.733826 8834 executor.go:111] Tasks: 48 done / 124 total; 31 can run I0512 07:43:07.734027 8834 iamrole.go:163] ListAttachedRolePolicies: masters.privatekopeio.example.com I0512 07:43:07.733968 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.734117 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I0512 07:43:07.734326 8834 iamrole.go:163] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0512 07:43:07.734383 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0512 07:43:07.734267 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0512 07:43:07.734575 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0512 07:43:07.734683 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.734915 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.735125 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.735457 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0512 07:43:07.735568 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.735821 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:07.735904 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0512 07:43:07.736099 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:07.736234 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] }] } I0512 07:43:07.736410 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.736585 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0512 07:43:07.736787 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.736967 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0512 07:43:07.737161 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:07.737289 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0512 07:43:07.737356 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0512 07:43:07.737424 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:43:07.737484 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0512 07:43:07.737532 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:07.737603 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0512 07:43:07.739050 8834 executor.go:111] Tasks: 79 done / 124 total; 39 can run I0512 07:43:07.739212 8834 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } W0512 07:43:07.739375 8834 api.go:53] PageSize not implemented I0512 07:43:07.739335 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:07.739425 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8","bastion-privatekopeio-exa-d8ef8e"] } I0512 07:43:07.739591 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0512 07:43:07.739499 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:07.739653 8834 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } W0512 07:43:07.739725 8834 api.go:53] PageSize not implemented I0512 07:43:07.739708 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:07.739760 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8","bastion-privatekopeio-exa-d8ef8e"] } I0512 07:43:07.739864 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0512 07:43:07.739909 8834 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0512 07:43:07.739924 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:07.740021 8834 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0512 07:43:07.740320 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:07.740431 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:07.740582 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:07.740642 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:07.740778 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0512 07:43:07.740865 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:07.740926 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:07.741040 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:07.743013 8834 executor.go:111] Tasks: 118 done / 124 total; 6 can run W0512 07:43:07.743132 8834 api.go:53] PageSize not implemented I0512 07:43:07.743127 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0512 07:43:07.743156 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8","bastion-privatekopeio-exa-d8ef8e"] } I0512 07:43:07.743248 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0512 07:43:07.743290 8834 api.go:53] PageSize not implemented I0512 07:43:07.743357 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8","bastion-privatekopeio-exa-d8ef8e"] } I0512 07:43:07.743243 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0512 07:43:07.743478 8834 dnsname.go:121] AliasTarget for "api.privatekopeio.example.com" is "api-privatekopeio-example-tl2bv8.elb.cloudmock.com" I0512 07:43:07.743509 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0512 07:43:07.743607 8834 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0512 07:43:07.747224 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0512 07:43:07.747290 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0512 07:43:07.747331 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0512 07:43:07.747369 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0512 07:43:07.747408 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0512 07:43:07.747444 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0512 07:43:07.747480 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0512 07:43:07.747515 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0512 07:43:07.747553 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0512 07:43:07.747590 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0512 07:43:07.747631 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0512 07:43:07.747671 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0512 07:43:07.747715 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0512 07:43:07.747752 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0512 07:43:07.747788 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0512 07:43:07.747854 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0512 07:43:07.747891 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0512 07:43:07.747931 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0512 07:43:07.747971 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0512 07:43:07.748009 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0512 07:43:07.748046 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0512 07:43:07.748083 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0512 07:43:07.748120 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0512 07:43:07.748157 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:43:07.748198 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0512 07:43:07.748233 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0512 07:43:07.748271 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0512 07:43:07.748311 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0512 07:43:07.748349 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0512 07:43:07.748382 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0512 07:43:07.748417 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0512 07:43:07.748456 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0512 07:43:07.748494 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0512 07:43:07.748530 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0512 07:43:07.750248 8834 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0512 07:43:07.750276 8834 keypairs.go:95] DescribeKeyPairs: { } I0512 07:43:07.750315 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.750385 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0512 07:43:07.750467 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.750533 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.750585 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.750638 8834 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.750679 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.750730 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0512 07:43:07.750799 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.750873 8834 routetable.go:53] DescribeRouteTables: { } I0512 07:43:07.750907 8834 natgateway.go:117] DescribeNatGateways: { } I0512 07:43:07.750948 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0512 07:43:07.750990 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } W0512 07:43:07.751043 8834 api.go:53] PageSize not implemented I0512 07:43:07.751061 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8","bastion-privatekopeio-exa-d8ef8e"] } I0512 07:43:07.751117 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0512 07:43:07.751164 8834 loadbalancers.go:34] PageSize not implemented I0512 07:43:07.751185 8834 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0512 07:43:07.751212 8834 targetgroups.go:35] PageSize not implemented I0512 07:43:07.751231 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0512 07:43:07.751281 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:07.751309 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:07.751358 8834 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0512 07:43:07.751397 8834 iamrole.go:91] ListRoles: { } I0512 07:43:07.751444 8834 internetgateways.go:101] DescribeInternetGateways: { } I0512 07:43:07.751495 8834 routetable.go:53] DescribeRouteTables: { } I0512 07:43:07.751546 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3","rtb-1","rtb-2"] } I0512 07:43:07.751592 8834 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0512 07:43:07.751626 8834 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0512 07:43:07.752254 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } autoscaling-group:bastion.privatekopeio.example.com ok autoscaling-group:nodes.privatekopeio.example.com ok I0512 07:43:07.752407 8834 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0512 07:43:07.752446 8834 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0512 07:43:07.752479 8834 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } iam-instance-profile:nodes.privatekopeio.example.com ok I0512 07:43:07.752323 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W0512 07:43:07.752552 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:07.752536 8834 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } autoscaling-group:master-us-test-1a.masters.privatekopeio.example.com ok I0512 07:43:07.752573 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok autoscaling-config:lt-2 ok I0512 07:43:07.752546 8834 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } load-balancer:bastion-privatekopeio-exa-d8ef8e ok I0512 07:43:07.752613 8834 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0512 07:43:07.752456 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:07.752637 8834 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.privatekopeio.example.com. ok load-balancer:api-privatekopeio-example-tl2bv8 ok I0512 07:43:07.752695 8834 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } iam-instance-profile:masters.privatekopeio.example.com ok I0512 07:43:07.752674 8834 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0512 07:43:07.752728 8834 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } autoscaling-config:lt-3 ok iam-instance-profile:bastions.privatekopeio.example.com ok I0512 07:43:07.752800 8834 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0512 07:43:07.752869 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:07.752933 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0512 07:43:07.753052 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:07.753069 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0512 07:43:07.753110 8834 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0512 07:43:07.753164 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0512 07:43:07.753218 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0512 07:43:07.753298 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:07.753319 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0512 07:43:07.753352 8834 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-1 ok I0512 07:43:07.753480 8834 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatekopeio.example.com" } I0512 07:43:07.753533 8834 iamrole.go:182] ListAttachedRolePolicies: masters.privatekopeio.example.com I0512 07:43:07.753503 8834 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0512 07:43:07.753548 8834 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0512 07:43:07.753607 8834 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-4" } subnet:subnet-4 ok I0512 07:43:07.753632 8834 iamrole.go:139] DeleteRole: { RoleName: "masters.privatekopeio.example.com" } iam-role:masters.privatekopeio.example.com ok I0512 07:43:07.753666 8834 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I0512 07:43:07.753693 8834 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatekopeio.example.com" } I0512 07:43:07.753720 8834 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0512 07:43:07.753757 8834 iamrole.go:182] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0512 07:43:07.753773 8834 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0512 07:43:07.753763 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:07.753807 8834 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatekopeio.example.com" } I0512 07:43:07.753855 8834 iamrole.go:182] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0512 07:43:07.753870 8834 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0512 07:43:07.753836 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0512 07:43:07.753928 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:07.753942 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0512 07:43:07.753903 8834 iamrole.go:139] DeleteRole: { RoleName: "nodes.privatekopeio.example.com" } iam-role:nodes.privatekopeio.example.com ok I0512 07:43:07.753981 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0512 07:43:07.754025 8834 iamrole.go:139] DeleteRole: { RoleName: "bastions.privatekopeio.example.com" } iam-role:bastions.privatekopeio.example.com ok I0512 07:43:07.754030 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W0512 07:43:07.754123 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:07.754136 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0512 07:43:07.754228 8834 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0512 07:43:07.754294 8834 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0512 07:43:07.754346 8834 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-3" } route-table:rtb-3 ok I0512 07:43:07.754431 8834 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0512 07:43:07.754539 8834 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0512 07:43:07.754805 8834 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateKopeio (0.42s) === RUN TestLifecycleIPv6 I0512 07:43:07.755309 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:43:07.755357 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:43:07.755399 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:43:07.755448 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:43:07.755488 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:43:07.755520 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:43:07.755552 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:43:07.755586 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:43:07.755613 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:43:07.755670 8834 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal-ipv6.example.com I0512 07:43:07.780415 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:07.783614 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:07.783707 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:07.820664 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:07.992937 8834 iamrole.go:91] ListRoles: { } I0512 07:43:07.994978 8834 executor.go:111] Tasks: 0 done / 105 total; 46 can run I0512 07:43:07.995487 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:07.995561 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:07.995620 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } I0512 07:43:07.995705 8834 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] } I0512 07:43:07.995776 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } I0512 07:43:07.995769 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:07.995829 8834 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" }] } I0512 07:43:07.995749 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0512 07:43:07.996000 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0512 07:43:07.997020 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] }] } I0512 07:43:07.997627 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0512 07:43:07.996006 8834 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0512 07:43:07.998005 8834 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0512 07:43:07.998105 8834 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0512 07:43:07.998152 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0512 07:43:07.998457 8834 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0512 07:43:07.998666 8834 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-main.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0512 07:43:07.999299 8834 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0512 07:43:07.999742 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0512 07:43:07.999878 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0512 07:43:08.000068 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0512 07:43:08.000226 8834 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0512 07:43:08.000608 8834 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0512 07:43:08.000664 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0512 07:43:08.000733 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:08.002759 8834 executor.go:111] Tasks: 46 done / 105 total; 21 can run I0512 07:43:08.002964 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0512 07:43:08.002914 8834 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0512 07:43:08.003077 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:08.003126 8834 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-minimal-ipv6-example--bne5ih", Port: 443, Protocol: "TCP", Tags: [{ Key: "Name", Value: "tcp-minimal-ipv6-example--bne5ih" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0512 07:43:08.003207 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0512 07:43:08.003251 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0512 07:43:08.003408 8834 iamrole.go:163] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0512 07:43:08.003393 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] }] } I0512 07:43:08.003544 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public-us-test-1a"] }] } I0512 07:43:08.003536 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0512 07:43:08.003636 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0512 07:43:08.003718 8834 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" }, { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0512 07:43:08.003824 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0512 07:43:08.003880 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0512 07:43:08.003882 8834 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0512 07:43:08.004013 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0512 07:43:08.004073 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:08.004124 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:08.004300 8834 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" } ] } I0512 07:43:08.004448 8834 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0512 07:43:08.004597 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0512 07:43:08.004823 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0512 07:43:08.004941 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0512 07:43:08.005011 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0512 07:43:08.005063 8834 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], VpcId: "vpc-1" } I0512 07:43:08.005253 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0512 07:43:08.005311 8834 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] } I0512 07:43:08.005396 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0512 07:43:08.005165 8834 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0512 07:43:08.005551 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0512 07:43:08.005657 8834 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0512 07:43:08.005713 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "api-elb.minimal-ipv6.example.com" }] }], VpcId: "vpc-1" } I0512 07:43:08.005815 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0512 07:43:08.005866 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0512 07:43:08.005907 8834 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0512 07:43:08.005989 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:08.006046 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0512 07:43:08.008124 8834 executor.go:111] Tasks: 67 done / 105 total; 30 can run I0512 07:43:08.008266 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:08.008457 8834 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0512 07:43:08.008442 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0512 07:43:08.008603 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0512 07:43:08.008836 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-udp-1to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0512 07:43:08.008987 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:08.009127 8834 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0512 07:43:08.009251 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0512 07:43:08.009435 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-1to2379-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0512 07:43:08.009606 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:08.009689 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0512 07:43:08.009786 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.009892 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0512 07:43:08.010027 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:08.010099 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0512 07:43:08.010228 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0512 07:43:08.010332 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:08.010442 8834 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0512 07:43:08.010427 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", Ipv6CidrBlock: "2001:db8:0:111::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], VpcId: "vpc-1" } I0512 07:43:08.010502 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" }] }] } I0512 07:43:08.010711 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0512 07:43:08.010863 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0512 07:43:08.010999 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0512 07:43:08.011163 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" }] }] } I0512 07:43:08.011307 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0512 07:43:08.011450 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.011606 8834 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0512 07:43:08.011700 8834 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0512 07:43:08.011749 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-2382to4000-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0512 07:43:08.011936 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:43:08.011997 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:43:08.012053 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0512 07:43:08.012246 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0512 07:43:08.012385 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0512 07:43:08.012487 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" }] }] } I0512 07:43:08.012626 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.012727 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.012807 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-4003to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0512 07:43:08.015761 8834 executor.go:111] Tasks: 97 done / 105 total; 5 can run I0512 07:43:08.015923 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0512 07:43:08.016016 8834 loadbalancers.go:34] PageSize not implemented I0512 07:43:08.015974 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:08.016085 8834 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:43:08.016162 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:08.016232 8834 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0512 07:43:08.016141 8834 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "dualstack", Name: "api-minimal-ipv6-example--jhj9te", SubnetMappings: [{ SubnetId: "subnet-1" }], Tags: [{ Key: "Name", Value: "api.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], Type: "network" } I0512 07:43:08.016330 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0512 07:43:08.016386 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:43:08.016369 8834 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1", Port: 443, Protocol: "TCP" } I0512 07:43:08.016433 8834 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0512 07:43:08.016459 8834 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "false" },{ Key: "access_logs.s3.enabled", Value: "false" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0512 07:43:08.016485 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0512 07:43:08.016549 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0512 07:43:08.016624 8834 executor.go:111] Tasks: 102 done / 105 total; 3 can run I0512 07:43:08.016698 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0512 07:43:08.016708 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:08.016800 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0512 07:43:08.016924 8834 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "64:ff9b::/96", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I0512 07:43:08.016981 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0512 07:43:08.017093 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0512 07:43:08.017205 8834 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0512 07:43:08.017274 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:08.017307 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:08.017365 8834 dns.go:238] Pre-creating DNS records I0512 07:43:08.017392 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:08.023683 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:08.026857 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.026955 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.065019 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:08.244941 8834 iamrole.go:91] ListRoles: { } I0512 07:43:08.247159 8834 executor.go:111] Tasks: 0 done / 105 total; 46 can run I0512 07:43:08.247502 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0512 07:43:08.247691 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0512 07:43:08.247755 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0512 07:43:08.247878 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:08.247955 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:08.247955 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] }] } I0512 07:43:08.248185 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0512 07:43:08.248351 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0512 07:43:08.248486 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:08.248561 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:08.250547 8834 executor.go:111] Tasks: 46 done / 105 total; 21 can run I0512 07:43:08.250690 8834 iamrole.go:163] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0512 07:43:08.250766 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0512 07:43:08.250752 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0512 07:43:08.250934 8834 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0512 07:43:08.250890 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0512 07:43:08.251039 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0512 07:43:08.251211 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:08.251313 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:08.251437 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0512 07:43:08.251585 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0512 07:43:08.251735 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:08.251786 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0512 07:43:08.252004 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0512 07:43:08.252065 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0512 07:43:08.252210 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:08.256081 8834 executor.go:111] Tasks: 67 done / 105 total; 30 can run I0512 07:43:08.256205 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:08.256300 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:08.256431 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.256550 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] }] } I0512 07:43:08.256914 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.257097 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.257279 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:08.257393 8834 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0512 07:43:08.257436 8834 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0512 07:43:08.257473 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0512 07:43:08.257611 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:08.257945 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:43:08.258052 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:08.258191 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.261842 8834 executor.go:111] Tasks: 97 done / 105 total; 5 can run I0512 07:43:08.262084 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } I0512 07:43:08.262103 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0512 07:43:08.262234 8834 loadbalancers.go:34] PageSize not implemented I0512 07:43:08.262271 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0512 07:43:08.262299 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:08.262368 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0512 07:43:08.262453 8834 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0512 07:43:08.262466 8834 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0512 07:43:08.262551 8834 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0512 07:43:08.262613 8834 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0512 07:43:08.262889 8834 executor.go:111] Tasks: 102 done / 105 total; 3 can run I0512 07:43:08.263042 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0512 07:43:08.263177 8834 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0512 07:43:08.263231 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0512 07:43:08.263325 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0512 07:43:08.263390 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0512 07:43:08.263434 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:08.263564 8834 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0512 07:43:08.263615 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0512 07:43:08.263686 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0512 07:43:08.263908 8834 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0512 07:43:08.268453 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0512 07:43:08.268526 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0512 07:43:08.268566 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0512 07:43:08.268601 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0512 07:43:08.268640 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0512 07:43:08.268680 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0512 07:43:08.268718 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0512 07:43:08.268752 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0512 07:43:08.268788 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0512 07:43:08.268826 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0512 07:43:08.268862 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0512 07:43:08.268902 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0512 07:43:08.268948 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0512 07:43:08.268988 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0512 07:43:08.269022 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0512 07:43:08.269080 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0512 07:43:08.269147 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0512 07:43:08.269210 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0512 07:43:08.269269 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0512 07:43:08.269336 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0512 07:43:08.269401 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0512 07:43:08.269467 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:43:08.269534 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0512 07:43:08.269594 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0512 07:43:08.269652 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0512 07:43:08.269716 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0512 07:43:08.269781 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0512 07:43:08.269860 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0512 07:43:08.269940 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0512 07:43:08.271557 8834 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0512 07:43:08.271587 8834 keypairs.go:95] DescribeKeyPairs: { } I0512 07:43:08.271628 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0512 07:43:08.271687 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0512 07:43:08.271753 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0512 07:43:08.271815 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0512 07:43:08.271869 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0512 07:43:08.271923 8834 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0512 07:43:08.271965 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0512 07:43:08.272011 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0512 07:43:08.272065 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0512 07:43:08.272114 8834 routetable.go:53] DescribeRouteTables: { } I0512 07:43:08.272142 8834 natgateway.go:117] DescribeNatGateways: { } I0512 07:43:08.272183 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0512 07:43:08.272225 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } W0512 07:43:08.272276 8834 api.go:53] PageSize not implemented I0512 07:43:08.272294 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0512 07:43:08.272337 8834 loadbalancers.go:34] PageSize not implemented I0512 07:43:08.272362 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0512 07:43:08.272438 8834 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0512 07:43:08.272483 8834 targetgroups.go:35] PageSize not implemented I0512 07:43:08.272514 8834 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1" ] } I0512 07:43:08.272620 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:08.272668 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:08.272808 8834 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0512 07:43:08.272866 8834 iamrole.go:91] ListRoles: { } I0512 07:43:08.272940 8834 internetgateways.go:101] DescribeInternetGateways: { } I0512 07:43:08.273022 8834 routetable.go:53] DescribeRouteTables: { } I0512 07:43:08.273101 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I0512 07:43:08.273178 8834 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0512 07:43:08.273244 8834 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0512 07:43:08.273837 8834 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0512 07:43:08.273921 8834 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } I0512 07:43:08.273977 8834 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0512 07:43:08.273891 8834 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0512 07:43:08.274041 8834 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } autoscaling-config:lt-2 ok I0512 07:43:08.273990 8834 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } I0512 07:43:08.274077 8834 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4 ok iam-instance-profile:nodes.minimal-ipv6.example.com ok I0512 07:43:08.274093 8834 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok internet-gateway:igw-2 ok I0512 07:43:08.274132 8834 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1 ok autoscaling-group:master-us-test-1a.masters.minimal-ipv6.example.com ok I0512 07:43:08.274161 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:08.274151 8834 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } autoscaling-group:nodes.minimal-ipv6.example.com ok I0512 07:43:08.274293 8834 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } iam-instance-profile:masters.minimal-ipv6.example.com ok I0512 07:43:08.274011 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [ { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }, { Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } } ] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/AAAA/api.minimal-ipv6.example.com ok route-table:rtb-2 ok I0512 07:43:08.274263 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0512 07:43:08.274642 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:08.274667 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-1 ok I0512 07:43:08.274720 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0512 07:43:08.274778 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0512 07:43:08.274809 8834 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0512 07:43:08.274864 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:08.274915 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0512 07:43:08.275047 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:08.275065 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0512 07:43:08.275107 8834 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0512 07:43:08.275215 8834 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal-ipv6.example.com" } I0512 07:43:08.275282 8834 iamrole.go:182] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0512 07:43:08.275309 8834 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0512 07:43:08.275282 8834 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } I0512 07:43:08.275362 8834 iamrole.go:139] DeleteRole: { RoleName: "nodes.minimal-ipv6.example.com" } elastic-ip:eipalloc-1 ok iam-role:nodes.minimal-ipv6.example.com ok I0512 07:43:08.275409 8834 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal-ipv6.example.com" } I0512 07:43:08.275477 8834 iamrole.go:182] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0512 07:43:08.275499 8834 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0512 07:43:08.275465 8834 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0512 07:43:08.275555 8834 iamrole.go:139] DeleteRole: { RoleName: "masters.minimal-ipv6.example.com" } iam-role:masters.minimal-ipv6.example.com ok I0512 07:43:08.275673 8834 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0512 07:43:08.275824 8834 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0512 07:43:08.275977 8834 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0512 07:43:08.276347 8834 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleIPv6 (0.52s) === RUN TestLifecycleSharedVPC I0512 07:43:08.276844 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:43:08.276890 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:43:08.276928 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:43:08.276972 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:43:08.277013 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:43:08.277044 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:43:08.277075 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:43:08.277109 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:43:08.277136 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:43:08.277190 8834 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedvpc.example.com I0512 07:43:08.301464 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:08.304197 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.304286 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.341798 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:08.452981 8834 iamrole.go:91] ListRoles: { } I0512 07:43:08.454455 8834 executor.go:111] Tasks: 0 done / 83 total; 43 can run I0512 07:43:08.454702 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:08.454849 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } I0512 07:43:08.454931 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0512 07:43:08.455034 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0512 07:43:08.455092 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0512 07:43:08.454949 8834 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0512 07:43:08.455177 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] }] } I0512 07:43:08.455429 8834 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedvpc.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0512 07:43:08.455578 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0512 07:43:08.455448 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } I0512 07:43:08.455635 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:08.455656 8834 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0512 07:43:08.455688 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:08.455841 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0512 07:43:08.455828 8834 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.sharedvpc.example.com", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "masters.sharedvpc.example.com" }] } I0512 07:43:08.456074 8834 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedvpc.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0512 07:43:08.456150 8834 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.sharedvpc.example.com", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.sharedvpc.example.com" }] } I0512 07:43:08.456241 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0512 07:43:08.456300 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0512 07:43:08.457867 8834 executor.go:111] Tasks: 43 done / 83 total; 17 can run I0512 07:43:08.458018 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0512 07:43:08.458022 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0512 07:43:08.458182 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0512 07:43:08.458223 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0512 07:43:08.458341 8834 iamrole.go:163] ListAttachedRolePolicies: masters.sharedvpc.example.com I0512 07:43:08.458409 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0512 07:43:08.458529 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0512 07:43:08.458634 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0512 07:43:08.458687 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-12345678" } I0512 07:43:08.458855 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0512 07:43:08.458965 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0512 07:43:08.459031 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:08.459078 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.sharedvpc.example.com" }] }], VpcId: "vpc-12345678" } I0512 07:43:08.459057 8834 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0512 07:43:08.459174 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0512 07:43:08.459220 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:08.459264 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0512 07:43:08.459435 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0512 07:43:08.459526 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:43:08.459586 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:43:08.459625 8834 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-12345678" } I0512 07:43:08.459779 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0512 07:43:08.459863 8834 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0512 07:43:08.460793 8834 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0512 07:43:08.463081 8834 executor.go:111] Tasks: 60 done / 83 total; 21 can run I0512 07:43:08.463290 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-::/0" }] }] } I0512 07:43:08.463627 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-masters.sharedvpc.example.com" }] }] } I0512 07:43:08.463828 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.463949 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.464112 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.464251 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.464331 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0512 07:43:08.464552 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0512 07:43:08.465750 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:08.466087 8834 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0512 07:43:08.466226 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:08.466395 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:08.466533 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-1to2379-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0512 07:43:08.466846 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0512 07:43:08.467145 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0512 07:43:08.467479 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-4003to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0512 07:43:08.467761 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-2382to4000-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0512 07:43:08.468438 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0512 07:43:08.470552 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0512 07:43:08.470632 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0512 07:43:08.470788 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0512 07:43:08.470929 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0512 07:43:08.471063 8834 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0512 07:43:08.471112 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-udp-1to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0512 07:43:08.471292 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0512 07:43:08.471387 8834 executor.go:111] Tasks: 81 done / 83 total; 2 can run I0512 07:43:08.471667 8834 executor.go:111] Tasks: 83 done / 83 total; 0 can run I0512 07:43:08.471729 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:08.471767 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:08.471830 8834 dns.go:238] Pre-creating DNS records I0512 07:43:08.471857 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:08.478332 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:08.481349 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.481497 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.519108 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:08.619063 8834 iamrole.go:91] ListRoles: { } I0512 07:43:08.620312 8834 executor.go:111] Tasks: 0 done / 83 total; 43 can run I0512 07:43:08.620540 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0512 07:43:08.622083 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0512 07:43:08.622164 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0512 07:43:08.622220 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:08.622455 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0512 07:43:08.622844 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0512 07:43:08.624297 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:08.624512 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:08.628224 8834 executor.go:111] Tasks: 43 done / 83 total; 17 can run I0512 07:43:08.628384 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0512 07:43:08.628579 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0512 07:43:08.628548 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0512 07:43:08.628713 8834 iamrole.go:163] ListAttachedRolePolicies: masters.sharedvpc.example.com I0512 07:43:08.628769 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:08.628868 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0512 07:43:08.629000 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0512 07:43:08.629157 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:08.629226 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0512 07:43:08.629462 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:43:08.635254 8834 executor.go:111] Tasks: 60 done / 83 total; 21 can run I0512 07:43:08.635481 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:08.635626 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.635911 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:08.636108 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.636276 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.636509 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:08.636701 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.638938 8834 executor.go:111] Tasks: 81 done / 83 total; 2 can run I0512 07:43:08.639163 8834 executor.go:111] Tasks: 83 done / 83 total; 0 can run I0512 07:43:08.642031 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0512 07:43:08.642097 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0512 07:43:08.642133 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0512 07:43:08.642169 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0512 07:43:08.642210 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0512 07:43:08.642244 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0512 07:43:08.642278 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0512 07:43:08.642315 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0512 07:43:08.642353 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0512 07:43:08.642406 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0512 07:43:08.642442 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0512 07:43:08.642476 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0512 07:43:08.642510 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0512 07:43:08.642543 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0512 07:43:08.642578 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0512 07:43:08.642613 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:43:08.642649 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0512 07:43:08.642684 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0512 07:43:08.642720 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0512 07:43:08.642754 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0512 07:43:08.642790 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0512 07:43:08.642826 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0512 07:43:08.644228 8834 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0512 07:43:08.644254 8834 keypairs.go:95] DescribeKeyPairs: { } I0512 07:43:08.644290 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0512 07:43:08.644340 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0512 07:43:08.644395 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0512 07:43:08.644453 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0512 07:43:08.644520 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0512 07:43:08.644561 8834 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0512 07:43:08.644600 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0512 07:43:08.644639 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0512 07:43:08.644685 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0512 07:43:08.644740 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0512 07:43:08.644778 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } W0512 07:43:08.644819 8834 api.go:53] PageSize not implemented I0512 07:43:08.644839 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0512 07:43:08.644865 8834 loadbalancers.go:34] PageSize not implemented I0512 07:43:08.644882 8834 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0512 07:43:08.644907 8834 targetgroups.go:35] PageSize not implemented I0512 07:43:08.644925 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0512 07:43:08.644989 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:08.645017 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:08.645064 8834 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0512 07:43:08.645102 8834 iamrole.go:91] ListRoles: { } I0512 07:43:08.645142 8834 internetgateways.go:101] DescribeInternetGateways: { } I0512 07:43:08.645180 8834 routetable.go:53] DescribeRouteTables: { } I0512 07:43:08.645208 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:08.645629 8834 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } autoscaling-config:lt-2 ok I0512 07:43:08.645692 8834 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } iam-instance-profile:masters.sharedvpc.example.com ok autoscaling-group:nodes.sharedvpc.example.com ok I0512 07:43:08.645750 8834 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } I0512 07:43:08.645751 8834 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0512 07:43:08.645822 8834 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } iam-instance-profile:nodes.sharedvpc.example.com ok I0512 07:43:08.645815 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } volume:vol-1 ok autoscaling-group:master-us-test-1a.masters.sharedvpc.example.com ok I0512 07:43:08.645852 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.sharedvpc.example.com. ok I0512 07:43:08.645912 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0512 07:43:08.646138 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:08.646167 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0512 07:43:08.646208 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:08.646265 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0512 07:43:08.646377 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:08.646392 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0512 07:43:08.646424 8834 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-1 ok I0512 07:43:08.646503 8834 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0512 07:43:08.646619 8834 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedvpc.example.com" } I0512 07:43:08.646672 8834 iamrole.go:182] ListAttachedRolePolicies: masters.sharedvpc.example.com I0512 07:43:08.646688 8834 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0512 07:43:08.646724 8834 iamrole.go:139] DeleteRole: { RoleName: "masters.sharedvpc.example.com" } iam-role:masters.sharedvpc.example.com ok I0512 07:43:08.646697 8834 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0512 07:43:08.646757 8834 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedvpc.example.com" } I0512 07:43:08.646800 8834 iamrole.go:182] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0512 07:43:08.646812 8834 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0512 07:43:08.646841 8834 iamrole.go:139] DeleteRole: { RoleName: "nodes.sharedvpc.example.com" } iam-role:nodes.sharedvpc.example.com ok I0512 07:43:08.646899 8834 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0512 07:43:08.647244 8834 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedVPC (0.37s) === RUN TestLifecycleComplex I0512 07:43:08.647782 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:43:08.647860 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:43:08.647903 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:43:08.647951 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:43:08.648012 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:43:08.648046 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:43:08.648077 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:43:08.648110 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:43:08.648138 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:43:08.648197 8834 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster complex.example.com I0512 07:43:08.688162 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:08.692498 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.692603 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.729484 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:08.842248 8834 iamrole.go:91] ListRoles: { } I0512 07:43:08.844448 8834 executor.go:111] Tasks: 0 done / 116 total; 48 can run I0512 07:43:08.844748 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } I0512 07:43:08.844767 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } I0512 07:43:08.844890 8834 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com", Tags: [ { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] } I0512 07:43:08.845021 8834 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/main", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-main.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0512 07:43:08.845209 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0512 07:43:08.845159 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0512 07:43:08.845309 8834 api.go:53] PageSize not implemented I0512 07:43:08.845296 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } I0512 07:43:08.845278 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0512 07:43:08.845439 8834 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", Tags: [ { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] } I0512 07:43:08.845558 8834 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "complex.example.com" } ] }] } I0512 07:43:08.845741 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0512 07:43:08.845630 8834 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", PermissionsBoundary: "arn:aws-test:iam::000000000000:policy/boundaries", RoleName: "nodes.complex.example.com", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] } I0512 07:43:08.845814 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0512 07:43:08.845923 8834 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", PermissionsBoundary: "arn:aws-test:iam::000000000000:policy/boundaries", RoleName: "masters.complex.example.com", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] } I0512 07:43:08.846012 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0512 07:43:08.846105 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0512 07:43:08.846200 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0512 07:43:08.846292 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:08.846486 8834 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "k8s.io/etcd/events", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-events.complex.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0512 07:43:08.847724 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:08.848180 8834 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0512 07:43:08.848327 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0512 07:43:08.848506 8834 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0512 07:43:08.848597 8834 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0512 07:43:08.848649 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0512 07:43:08.848743 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:08.850235 8834 executor.go:111] Tasks: 48 done / 116 total; 27 can run I0512 07:43:08.850452 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:08.850747 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0512 07:43:08.850822 8834 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0512 07:43:08.850927 8834 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0512 07:43:08.850823 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0512 07:43:08.850970 8834 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0512 07:43:08.851044 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0512 07:43:08.851275 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0512 07:43:08.851128 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0512 07:43:08.851277 8834 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tls-complex-example-com-5nursn", Port: 443, Protocol: "TLS", Tags: [ { Key: "Name", Value: "tls-complex-example-com-5nursn" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0512 07:43:08.851335 8834 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0512 07:43:08.853013 8834 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0512 07:43:08.852100 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0512 07:43:08.852307 8834 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-complex-example-com-vpjolq", Port: 443, Protocol: "TCP", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "tcp-complex-example-com-vpjolq" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0512 07:43:08.853268 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "api-elb.complex.example.com" } ] }], VpcId: "vpc-1" } I0512 07:43:08.853394 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:08.854234 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:08.854351 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:08.854429 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:08.854517 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0512 07:43:08.854621 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:08.854589 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-private.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I0512 07:43:08.854710 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:08.854813 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:08.854903 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:08.854973 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:08.854906 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.96.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-utility.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I0512 07:43:08.855052 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "masters.complex.example.com" } ] }], VpcId: "vpc-1" } I0512 07:43:08.855162 8834 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0512 07:43:08.855233 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0512 07:43:08.855159 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.complex.example.com" } ] }], VpcId: "vpc-1" } I0512 07:43:08.855325 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0512 07:43:08.855385 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0512 07:43:08.855366 8834 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0512 07:43:08.855450 8834 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.1.0.0/16", VpcId: "vpc-1" } I0512 07:43:08.855548 8834 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0512 07:43:08.855597 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0512 07:43:08.855719 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0512 07:43:08.855783 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0512 07:43:08.855832 8834 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.2.0.0/16", VpcId: "vpc-1" } I0512 07:43:08.855886 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:43:08.855935 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:43:08.855978 8834 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0512 07:43:08.855704 8834 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0512 07:43:08.856079 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0512 07:43:08.856131 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0512 07:43:08.856188 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0512 07:43:08.856236 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0512 07:43:08.856282 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0512 07:43:08.856330 8834 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" } ] } I0512 07:43:08.856423 8834 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "complex.example.com" } ] } I0512 07:43:08.856510 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:08.856553 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0512 07:43:08.856602 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0512 07:43:08.856643 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:08.859651 8834 executor.go:111] Tasks: 75 done / 116 total; 38 can run I0512 07:43:08.859933 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-4003to65535-masters.complex.example.com" } ] }] } I0512 07:43:08.860052 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0512 07:43:08.860134 8834 loadbalancers.go:34] PageSize not implemented I0512 07:43:08.860218 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0512 07:43:08.860245 8834 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "api-complex-example-com-vd3t5n", SubnetMappings: [{ AllocationId: "eipalloc-012345a678b9cdefa", SubnetId: "subnet-3" }], Tags: [ { Key: "Name", Value: "api.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ], Type: "network" } I0512 07:43:08.860373 8834 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-test-1:000000000000:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0512 07:43:08.860465 8834 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0512 07:43:08.860413 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }] } I0512 07:43:08.860530 8834 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [ { Key: "load_balancing.cross_zone.enabled", Value: "true" }, { Key: "access_logs.s3.enabled", Value: "true" }, { Key: "access_logs.s3.bucket", Value: "access-log-example" }, { Key: "access_logs.s3.prefix", Value: "" } ], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0512 07:43:08.860692 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-udp-1to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0512 07:43:08.860894 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0512 07:43:08.861208 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.0/24-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0512 07:43:08.861394 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:08.861498 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:08.861586 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:08.861665 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:08.861831 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0512 07:43:08.862067 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:08.862315 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0512 07:43:08.862722 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }] } I0512 07:43:08.862837 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0512 07:43:08.862929 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0512 07:43:08.863080 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-1to2379-masters.complex.example.com" } ] }] } I0512 07:43:08.863235 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0512 07:43:08.863395 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0512 07:43:08.863549 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.863654 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-2382to4000-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0512 07:43:08.863818 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.complex.example.com" } ] }] } I0512 07:43:08.863964 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0512 07:43:08.864106 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0512 07:43:08.864249 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0512 07:43:08.864385 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }] } I0512 07:43:08.864484 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }] } I0512 07:43:08.864639 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0512 07:43:08.864937 8834 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", RouteTableId: "rtb-2", TransitGatewayId: "tgw-123456" } I0512 07:43:08.865052 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.865216 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0512 07:43:08.865292 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0512 07:43:08.865576 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0512 07:43:08.865619 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:08.865695 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0512 07:43:08.865731 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0512 07:43:08.865822 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-::/0" } ] }] } I0512 07:43:08.865962 8834 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0512 07:43:08.866008 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.866056 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }] } I0512 07:43:08.866153 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0512 07:43:08.866247 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-3" } I0512 07:43:08.866306 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.866449 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0512 07:43:08.866536 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0512 07:43:08.866604 8834 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0512 07:43:08.868865 8834 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0512 07:43:08.869070 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0512 07:43:08.869239 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0512 07:43:08.869366 8834 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0512 07:43:08.869418 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:08.869455 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:08.869509 8834 dns.go:238] Pre-creating DNS records I0512 07:43:08.869533 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:08.877103 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:08.880863 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.880964 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:08.919024 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:09.031332 8834 iamrole.go:91] ListRoles: { } I0512 07:43:09.033590 8834 executor.go:111] Tasks: 0 done / 116 total; 48 can run I0512 07:43:09.033862 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0512 07:43:09.034045 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0512 07:43:09.034217 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0512 07:43:09.034321 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0512 07:43:09.034446 8834 api.go:53] PageSize not implemented I0512 07:43:09.034369 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.034654 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0512 07:43:09.034850 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0512 07:43:09.035076 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0512 07:43:09.035135 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0512 07:43:09.035240 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.035320 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.035533 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.037028 8834 executor.go:111] Tasks: 48 done / 116 total; 27 can run I0512 07:43:09.037227 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.037376 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.037468 8834 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0512 07:43:09.037529 8834 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0512 07:43:09.037612 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0512 07:43:09.037536 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.037643 8834 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0512 07:43:09.037732 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0512 07:43:09.037757 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0512 07:43:09.037828 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0512 07:43:09.037782 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0512 07:43:09.037961 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0512 07:43:09.038040 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] }] } I0512 07:43:09.038181 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0512 07:43:09.038222 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0512 07:43:09.038400 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:09.038448 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.038561 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.038646 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.038713 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.038821 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.038889 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.038952 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:43:09.038995 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0512 07:43:09.039175 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:09.042174 8834 executor.go:111] Tasks: 75 done / 116 total; 38 can run I0512 07:43:09.042365 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:09.042461 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0512 07:43:09.042538 8834 loadbalancers.go:34] PageSize not implemented I0512 07:43:09.042555 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:09.042544 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:09.042598 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:09.042676 8834 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0512 07:43:09.042731 8834 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0512 07:43:09.042765 8834 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0512 07:43:09.042799 8834 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0512 07:43:09.042820 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.043028 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.043534 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:09.043727 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.043892 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:09.044026 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:09.044120 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:09.044232 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.045462 8834 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0512 07:43:09.045580 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0512 07:43:09.045679 8834 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0512 07:43:09.045707 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0512 07:43:09.045730 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:09.045824 8834 executor.go:111] Tasks: 116 done / 116 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.sslCertificate=arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678] I0512 07:43:09.056633 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:09.059982 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.073079 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:09.077299 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.077408 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.114506 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:09.219517 8834 iamrole.go:91] ListRoles: { } I0512 07:43:09.221793 8834 executor.go:111] Tasks: 0 done / 116 total; 48 can run I0512 07:43:09.221997 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } W0512 07:43:09.222168 8834 api.go:53] PageSize not implemented I0512 07:43:09.222269 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0512 07:43:09.222403 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.222624 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0512 07:43:09.222599 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:09.222749 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.222972 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0512 07:43:09.223080 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0512 07:43:09.223201 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0512 07:43:09.223231 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0512 07:43:09.223279 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.223464 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.225149 8834 executor.go:111] Tasks: 48 done / 116 total; 27 can run I0512 07:43:09.225339 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.225507 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0512 07:43:09.225624 8834 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0512 07:43:09.225525 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.225550 8834 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0512 07:43:09.225754 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0512 07:43:09.225779 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0512 07:43:09.225816 8834 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0512 07:43:09.225869 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.225917 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0512 07:43:09.226000 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.226144 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.226287 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.226509 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0512 07:43:09.226579 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0512 07:43:09.226745 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:09.226795 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0512 07:43:09.226949 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:09.227011 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:43:09.227114 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.227353 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.227564 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.227746 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.227902 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0512 07:43:09.228055 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0512 07:43:09.230323 8834 executor.go:111] Tasks: 75 done / 116 total; 38 can run I0512 07:43:09.230481 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0512 07:43:09.230555 8834 loadbalancers.go:34] PageSize not implemented I0512 07:43:09.230519 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:09.230572 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:09.230621 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:09.230614 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:09.230659 8834 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0512 07:43:09.230748 8834 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0512 07:43:09.230734 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:09.230798 8834 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0512 07:43:09.230847 8834 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0512 07:43:09.231039 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { Names: ["api-complex-example-com-vd3t5n"] } I0512 07:43:09.231013 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:09.231079 8834 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0512 07:43:09.231113 8834 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/1" } I0512 07:43:09.231169 8834 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/2" } I0512 07:43:09.231203 8834 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0512 07:43:09.231257 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:09.231296 8834 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0512 07:43:09.231362 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:09.231396 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:09.231522 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.231683 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.231758 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:09.231954 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.232149 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.234430 8834 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0512 07:43:09.234563 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0512 07:43:09.234661 8834 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0512 07:43:09.234688 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0512 07:43:09.234714 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:09.234807 8834 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0512 07:43:09.234860 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:09.234892 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:09.234945 8834 dns.go:238] Pre-creating DNS records I0512 07:43:09.242436 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:09.246157 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.246256 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.284345 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:09.390495 8834 iamrole.go:91] ListRoles: { } I0512 07:43:09.392761 8834 executor.go:111] Tasks: 0 done / 116 total; 48 can run I0512 07:43:09.392959 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.393160 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.393385 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0512 07:43:09.393523 8834 api.go:53] PageSize not implemented I0512 07:43:09.393566 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0512 07:43:09.393674 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:09.393655 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0512 07:43:09.393877 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0512 07:43:09.393989 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.394115 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0512 07:43:09.394150 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0512 07:43:09.394198 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0512 07:43:09.394406 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.395430 8834 executor.go:111] Tasks: 48 done / 116 total; 27 can run I0512 07:43:09.395600 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.395683 8834 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0512 07:43:09.395760 8834 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0512 07:43:09.395847 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0512 07:43:09.395858 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0512 07:43:09.395734 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0512 07:43:09.395942 8834 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0512 07:43:09.396004 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0512 07:43:09.396006 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0512 07:43:09.396206 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:09.396264 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.396359 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.396494 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.396581 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.396664 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.396790 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.396936 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:43:09.396977 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.397101 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0512 07:43:09.397156 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.397351 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0512 07:43:09.397518 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0512 07:43:09.397608 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:09.397739 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] }] } I0512 07:43:09.397949 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0512 07:43:09.400678 8834 executor.go:111] Tasks: 75 done / 116 total; 38 can run I0512 07:43:09.400858 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:09.400995 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:09.401093 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:09.401179 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0512 07:43:09.401271 8834 loadbalancers.go:34] PageSize not implemented I0512 07:43:09.401291 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:09.401338 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:09.401373 8834 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0512 07:43:09.401413 8834 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0512 07:43:09.401447 8834 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0512 07:43:09.401478 8834 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0512 07:43:09.401638 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.401818 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.401901 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:09.402090 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:09.402288 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:09.402535 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.402767 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.405094 8834 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0512 07:43:09.405213 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0512 07:43:09.405314 8834 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0512 07:43:09.405345 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0512 07:43:09.405376 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:09.405465 8834 executor.go:111] Tasks: 116 done / 116 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.additionalSecurityGroups=sg-123456] I0512 07:43:09.409112 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:09.412516 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.428119 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:09.431833 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.431937 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.469483 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:09.571857 8834 iamrole.go:91] ListRoles: { } I0512 07:43:09.573703 8834 executor.go:111] Tasks: 0 done / 117 total; 49 can run I0512 07:43:09.573860 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0512 07:43:09.574095 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:09.574118 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } W0512 07:43:09.576119 8834 api.go:53] PageSize not implemented I0512 07:43:09.576927 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0512 07:43:09.577091 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.577253 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0512 07:43:09.577578 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0512 07:43:09.577750 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0512 07:43:09.577942 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.578551 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.578869 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0512 07:43:09.578940 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0512 07:43:09.580964 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.582430 8834 executor.go:111] Tasks: 49 done / 117 total; 27 can run I0512 07:43:09.582571 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.582674 8834 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0512 07:43:09.582666 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.582763 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0512 07:43:09.582931 8834 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0512 07:43:09.582877 8834 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0512 07:43:09.582976 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0512 07:43:09.582828 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0512 07:43:09.582974 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0512 07:43:09.583116 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:09.583222 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.583373 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.583621 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0512 07:43:09.583701 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0512 07:43:09.583919 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:09.583993 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.584098 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0512 07:43:09.584172 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.584307 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.584453 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.584630 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0512 07:43:09.584798 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.584943 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.585062 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:43:09.585124 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0512 07:43:09.587968 8834 executor.go:111] Tasks: 76 done / 117 total; 38 can run I0512 07:43:09.588123 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.588403 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:09.588514 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0512 07:43:09.588600 8834 loadbalancers.go:34] PageSize not implemented I0512 07:43:09.588618 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:09.588519 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:09.588673 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:09.588708 8834 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0512 07:43:09.588747 8834 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0512 07:43:09.588784 8834 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0512 07:43:09.588820 8834 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0512 07:43:09.588882 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:09.589107 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:09.589357 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.589510 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:09.589783 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.589913 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:09.590116 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.592912 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.592990 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.595724 8834 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0512 07:43:09.595914 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0512 07:43:09.596015 8834 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0512 07:43:09.596038 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0512 07:43:09.596067 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:09.596157 8834 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0512 07:43:09.596220 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:09.596249 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:09.596306 8834 dns.go:238] Pre-creating DNS records I0512 07:43:09.604068 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:09.607835 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.607936 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.644847 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:09.752223 8834 iamrole.go:91] ListRoles: { } I0512 07:43:09.754445 8834 executor.go:111] Tasks: 0 done / 117 total; 49 can run I0512 07:43:09.754670 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:09.754790 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.755016 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } W0512 07:43:09.755229 8834 api.go:53] PageSize not implemented I0512 07:43:09.755203 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0512 07:43:09.755323 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0512 07:43:09.755439 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.755653 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.755793 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0512 07:43:09.755900 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0512 07:43:09.755999 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.756096 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0512 07:43:09.756126 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0512 07:43:09.756186 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.757871 8834 executor.go:111] Tasks: 49 done / 117 total; 27 can run I0512 07:43:09.758017 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.758223 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0512 07:43:09.758180 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0512 07:43:09.758349 8834 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0512 07:43:09.758240 8834 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0512 07:43:09.758353 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.758435 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0512 07:43:09.758539 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.758565 8834 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0512 07:43:09.758661 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0512 07:43:09.758707 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0512 07:43:09.758749 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.758898 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0512 07:43:09.758941 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0512 07:43:09.759105 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:09.759157 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0512 07:43:09.759284 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.759373 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0512 07:43:09.759528 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:09.759591 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.759724 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:43:09.759775 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0512 07:43:09.759985 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0512 07:43:09.760085 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.760236 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.763086 8834 executor.go:111] Tasks: 76 done / 117 total; 38 can run I0512 07:43:09.763370 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0512 07:43:09.763478 8834 loadbalancers.go:34] PageSize not implemented I0512 07:43:09.763424 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.763505 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:09.763578 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:09.763638 8834 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0512 07:43:09.763606 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:09.763697 8834 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0512 07:43:09.763738 8834 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0512 07:43:09.763771 8834 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0512 07:43:09.763769 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:09.763973 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.764076 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:09.764287 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:09.764601 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:09.764742 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:09.764833 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.765169 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.767308 8834 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0512 07:43:09.767405 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0512 07:43:09.767493 8834 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0512 07:43:09.767518 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0512 07:43:09.767548 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:09.767630 8834 executor.go:111] Tasks: 117 done / 117 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t3.large] I0512 07:43:09.772641 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:09.775925 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.776022 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.789228 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:09.793372 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.793479 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.793544 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.831594 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:09.933698 8834 iamrole.go:91] ListRoles: { } I0512 07:43:09.935985 8834 executor.go:111] Tasks: 0 done / 117 total; 49 can run I0512 07:43:09.936368 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0512 07:43:09.936525 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:09.936496 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0512 07:43:09.936672 8834 api.go:53] PageSize not implemented I0512 07:43:09.936745 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.937017 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0512 07:43:09.937324 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0512 07:43:09.937488 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.937669 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.937787 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0512 07:43:09.937877 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0512 07:43:09.937950 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0512 07:43:09.938215 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0512 07:43:09.938478 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.942250 8834 executor.go:111] Tasks: 49 done / 117 total; 27 can run I0512 07:43:09.942428 8834 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0512 07:43:09.942514 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0512 07:43:09.942410 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0512 07:43:09.942579 8834 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0512 07:43:09.942627 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0512 07:43:09.942639 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.942740 8834 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0512 07:43:09.942855 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0512 07:43:09.942797 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.942990 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0512 07:43:09.943050 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0512 07:43:09.943133 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.943262 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0512 07:43:09.943446 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:09.943589 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.943675 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I0512 07:43:09.943794 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:43:09.943828 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.943904 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:09.943976 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.944093 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0512 07:43:09.944128 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0512 07:43:09.944259 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:09.944365 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0512 07:43:09.944527 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:09.947486 8834 executor.go:111] Tasks: 76 done / 117 total; 38 can run I0512 07:43:09.947664 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.947851 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.947825 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0512 07:43:09.947944 8834 loadbalancers.go:34] PageSize not implemented I0512 07:43:09.947970 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:09.948052 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:09.948119 8834 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0512 07:43:09.948099 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:09.948177 8834 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0512 07:43:09.948220 8834 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0512 07:43:09.948253 8834 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0512 07:43:09.948279 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:09.948496 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.948706 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.948963 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:09.949087 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:09.949219 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:09.949312 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:09.951296 8834 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0512 07:43:09.951469 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0512 07:43:09.951566 8834 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0512 07:43:09.951591 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0512 07:43:09.951622 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:09.951710 8834 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0512 07:43:09.951776 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:09.951810 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:09.951868 8834 dns.go:238] Pre-creating DNS records I0512 07:43:09.960260 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:09.964181 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.964278 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:09.964339 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.001353 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:10.110062 8834 iamrole.go:91] ListRoles: { } I0512 07:43:10.112381 8834 executor.go:111] Tasks: 0 done / 117 total; 49 can run I0512 07:43:10.112632 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0512 07:43:10.112809 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:10.112875 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0512 07:43:10.113227 8834 api.go:53] PageSize not implemented I0512 07:43:10.113176 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:10.113353 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0512 07:43:10.113412 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0512 07:43:10.113511 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:10.113582 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0512 07:43:10.113754 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0512 07:43:10.113921 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0512 07:43:10.114078 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] }] } I0512 07:43:10.114317 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0512 07:43:10.114562 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0512 07:43:10.115202 8834 executor.go:111] Tasks: 49 done / 117 total; 27 can run I0512 07:43:10.115336 8834 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0512 07:43:10.115331 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:10.115520 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0512 07:43:10.115426 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0512 07:43:10.115508 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0512 07:43:10.115631 8834 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0512 07:43:10.115724 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0512 07:43:10.115706 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:10.115807 8834 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0512 07:43:10.115873 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0512 07:43:10.115919 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:43:10.115962 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:10.116062 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0512 07:43:10.116218 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:10.116319 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:10.116447 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] }] } I0512 07:43:10.116582 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0512 07:43:10.116621 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0512 07:43:10.116774 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0512 07:43:10.116841 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:10.117009 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0512 07:43:10.117296 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:10.117368 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:10.117491 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:10.117651 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:10.120502 8834 executor.go:111] Tasks: 76 done / 117 total; 38 can run I0512 07:43:10.120641 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0512 07:43:10.120745 8834 loadbalancers.go:34] PageSize not implemented I0512 07:43:10.120719 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:10.120770 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:10.120837 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:10.120902 8834 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0512 07:43:10.120971 8834 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0512 07:43:10.120895 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:10.121034 8834 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0512 07:43:10.121074 8834 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0512 07:43:10.121072 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:10.121344 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:10.121541 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.121717 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.122003 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:10.122143 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:10.122409 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.122643 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.125843 8834 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0512 07:43:10.126342 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0512 07:43:10.127013 8834 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0512 07:43:10.127074 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0512 07:43:10.127111 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:10.127238 8834 executor.go:111] Tasks: 117 done / 117 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t2.medium] I0512 07:43:10.130836 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:10.134109 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.134203 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.134258 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.144612 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:10.148763 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.148861 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.148921 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.148978 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.185838 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:10.293900 8834 iamrole.go:91] ListRoles: { } I0512 07:43:10.296176 8834 executor.go:111] Tasks: 0 done / 117 total; 49 can run I0512 07:43:10.296460 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0512 07:43:10.296859 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:10.296960 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0512 07:43:10.297242 8834 api.go:53] PageSize not implemented I0512 07:43:10.297206 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0512 07:43:10.297395 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:10.297564 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0512 07:43:10.297610 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] }] } I0512 07:43:10.297995 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0512 07:43:10.298180 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0512 07:43:10.298352 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:10.298577 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0512 07:43:10.298668 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:10.298721 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0512 07:43:10.300304 8834 executor.go:111] Tasks: 49 done / 117 total; 27 can run I0512 07:43:10.300456 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:10.300564 8834 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0512 07:43:10.300658 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0512 07:43:10.300758 8834 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0512 07:43:10.300701 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:10.300818 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0512 07:43:10.300828 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0512 07:43:10.300845 8834 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0512 07:43:10.300972 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:10.301013 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0512 07:43:10.301108 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:43:10.301186 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:10.301339 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:10.301485 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:10.301634 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0512 07:43:10.301678 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:10.301764 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0512 07:43:10.301892 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:10.302022 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0512 07:43:10.302079 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0512 07:43:10.302302 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:10.302359 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0512 07:43:10.302633 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0512 07:43:10.302724 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:10.302863 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:10.306110 8834 executor.go:111] Tasks: 76 done / 117 total; 38 can run I0512 07:43:10.306326 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:10.306399 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } I0512 07:43:10.306429 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0512 07:43:10.306490 8834 loadbalancers.go:34] PageSize not implemented I0512 07:43:10.306511 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:10.306559 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:10.306630 8834 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0512 07:43:10.306623 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:10.306672 8834 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0512 07:43:10.306810 8834 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0512 07:43:10.306848 8834 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0512 07:43:10.306982 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:10.307088 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:10.307183 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:10.307378 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.307539 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.307785 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.308139 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.310639 8834 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0512 07:43:10.310729 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0512 07:43:10.310812 8834 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0512 07:43:10.310842 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0512 07:43:10.310865 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:10.311028 8834 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0512 07:43:10.311094 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:10.311132 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:10.311219 8834 dns.go:238] Pre-creating DNS records I0512 07:43:10.318911 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:10.322668 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.322767 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.322828 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.322882 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.360955 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:10.466372 8834 iamrole.go:91] ListRoles: { } I0512 07:43:10.468635 8834 executor.go:111] Tasks: 0 done / 117 total; 49 can run I0512 07:43:10.468919 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:10.469133 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0512 07:43:10.469193 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0512 07:43:10.469277 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:10.469288 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0512 07:43:10.469437 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0512 07:43:10.469581 8834 api.go:53] PageSize not implemented I0512 07:43:10.469656 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:10.469818 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0512 07:43:10.469931 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0512 07:43:10.470028 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:10.470075 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0512 07:43:10.470166 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:10.470304 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0512 07:43:10.471183 8834 executor.go:111] Tasks: 49 done / 117 total; 27 can run I0512 07:43:10.471337 8834 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0512 07:43:10.471416 8834 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0512 07:43:10.471427 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:10.471564 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:10.471501 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0512 07:43:10.471642 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0512 07:43:10.471677 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0512 07:43:10.471741 8834 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0512 07:43:10.471874 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:10.471902 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0512 07:43:10.471931 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] }] } I0512 07:43:10.472095 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0512 07:43:10.472144 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:10.472315 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:10.472409 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:43:10.472455 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:10.472550 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:10.472682 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:10.472815 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0512 07:43:10.472965 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0512 07:43:10.473020 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:10.473089 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:10.473192 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0512 07:43:10.473353 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:10.473483 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0512 07:43:10.476628 8834 executor.go:111] Tasks: 76 done / 117 total; 38 can run I0512 07:43:10.476844 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:10.476996 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:10.477036 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0512 07:43:10.477125 8834 loadbalancers.go:34] PageSize not implemented I0512 07:43:10.477117 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:10.477167 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:10.477217 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:10.477218 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:10.477258 8834 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0512 07:43:10.477300 8834 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0512 07:43:10.477341 8834 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0512 07:43:10.477372 8834 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0512 07:43:10.477528 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.477704 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.478111 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:10.478382 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:10.478575 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.478754 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.481319 8834 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0512 07:43:10.481436 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0512 07:43:10.481529 8834 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0512 07:43:10.481555 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0512 07:43:10.481583 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:10.481673 8834 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0512 07:43:10.481969 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0512 07:43:10.482026 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0512 07:43:10.482062 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0512 07:43:10.482096 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0512 07:43:10.482137 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0512 07:43:10.482174 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0512 07:43:10.482207 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0512 07:43:10.482242 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0512 07:43:10.482279 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0512 07:43:10.482321 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0512 07:43:10.482366 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0512 07:43:10.482401 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0512 07:43:10.482435 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0512 07:43:10.482474 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0512 07:43:10.482507 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0512 07:43:10.482543 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0512 07:43:10.482581 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0512 07:43:10.482618 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0512 07:43:10.482655 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:43:10.482695 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0512 07:43:10.482729 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0512 07:43:10.482770 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0512 07:43:10.482809 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0512 07:43:10.482842 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0512 07:43:10.482876 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0512 07:43:10.482915 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0512 07:43:10.482953 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0512 07:43:10.482993 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0512 07:43:10.485607 8834 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0512 07:43:10.485639 8834 keypairs.go:95] DescribeKeyPairs: { } I0512 07:43:10.485675 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:10.485734 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0512 07:43:10.485799 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:10.485860 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:10.485908 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:10.485966 8834 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:10.486008 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:10.486055 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0512 07:43:10.486112 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:10.486184 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0512 07:43:10.486225 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } W0512 07:43:10.486277 8834 api.go:53] PageSize not implemented I0512 07:43:10.486295 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0512 07:43:10.486321 8834 loadbalancers.go:34] PageSize not implemented I0512 07:43:10.486335 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0512 07:43:10.486373 8834 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0512 07:43:10.486396 8834 targetgroups.go:35] PageSize not implemented I0512 07:43:10.486413 8834 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" ] } I0512 07:43:10.486489 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:10.486519 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:10.486560 8834 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0512 07:43:10.486593 8834 iamrole.go:91] ListRoles: { } I0512 07:43:10.486634 8834 internetgateways.go:101] DescribeInternetGateways: { } I0512 07:43:10.486685 8834 routetable.go:53] DescribeRouteTables: { } I0512 07:43:10.486732 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I0512 07:43:10.487300 8834 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0512 07:43:10.487364 8834 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } iam-instance-profile:nodes.complex.example.com ok I0512 07:43:10.487335 8834 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1 ok I0512 07:43:10.487406 8834 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } autoscaling-group:nodes.complex.example.com ok I0512 07:43:10.487440 8834 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5 ok autoscaling-config:lt-2 ok autoscaling-group:master-us-test-1a.masters.complex.example.com ok volume:vol-1 ok I0512 07:43:10.487531 8834 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0512 07:43:10.487617 8834 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } I0512 07:43:10.487603 8834 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } volume:vol-2 ok I0512 07:43:10.487470 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:10.487673 8834 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } autoscaling-config:lt-1 ok I0512 07:43:10.487694 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:10.487726 8834 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4 ok I0512 07:43:10.487759 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0512 07:43:10.487825 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } route53-record:Z1AFAKE1ZON3YO/A/api.complex.example.com ok iam-instance-profile:masters.complex.example.com ok I0512 07:43:10.487887 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 } ] } W0512 07:43:10.488272 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:10.488294 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0512 07:43:10.488330 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0512 07:43:10.488392 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 } ] } W0512 07:43:10.488604 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:10.488636 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0512 07:43:10.488672 8834 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0512 07:43:10.488742 8834 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0512 07:43:10.488791 8834 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0512 07:43:10.488899 8834 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0512 07:43:10.488948 8834 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.complex.example.com" } I0512 07:43:10.489036 8834 iamrole.go:182] ListAttachedRolePolicies: nodes.complex.example.com I0512 07:43:10.489003 8834 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I0512 07:43:10.489057 8834 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.complex.example.com" } I0512 07:43:10.489123 8834 iamrole.go:182] ListAttachedRolePolicies: masters.complex.example.com I0512 07:43:10.489145 8834 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0512 07:43:10.489179 8834 iamrole.go:139] DeleteRole: { RoleName: "masters.complex.example.com" } iam-role:masters.complex.example.com ok I0512 07:43:10.489215 8834 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0512 07:43:10.489278 8834 iamrole.go:139] DeleteRole: { RoleName: "nodes.complex.example.com" } iam-role:nodes.complex.example.com ok I0512 07:43:10.489376 8834 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0512 07:43:10.489476 8834 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0512 07:43:10.489562 8834 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0512 07:43:10.489845 8834 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleComplex (1.84s) === RUN TestLifecycleExternalLB I0512 07:43:10.490352 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:43:10.490400 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:43:10.490447 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:43:10.490495 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:43:10.490535 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:43:10.490565 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:43:10.490597 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:43:10.490630 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:43:10.490658 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:43:10.490716 8834 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster externallb.example.com I0512 07:43:10.521040 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:10.523835 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.523940 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.561193 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:10.670701 8834 iamrole.go:91] ListRoles: { } I0512 07:43:10.672811 8834 executor.go:111] Tasks: 0 done / 93 total; 51 can run I0512 07:43:10.673073 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:10.673141 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0512 07:43:10.673367 8834 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.externallb.example.com", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } W0512 07:43:10.673608 8834 api.go:53] PageSize not implemented I0512 07:43:10.673526 8834 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0512 07:43:10.673425 8834 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0512 07:43:10.673675 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0512 07:43:10.673711 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } W0512 07:43:10.673776 8834 api.go:53] PageSize not implemented I0512 07:43:10.673684 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } I0512 07:43:10.673744 8834 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } W0512 07:43:10.673892 8834 api.go:53] PageSize not implemented I0512 07:43:10.673795 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0512 07:43:10.673866 8834 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I0512 07:43:10.673931 8834 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0512 07:43:10.674061 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0512 07:43:10.674134 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0512 07:43:10.674094 8834 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "externallb.example.com" }] }] } I0512 07:43:10.674046 8834 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.externallb.example.com", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] } I0512 07:43:10.674277 8834 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0512 07:43:10.674338 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:10.674401 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } I0512 07:43:10.674316 8834 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0512 07:43:10.674486 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:10.674549 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0512 07:43:10.674534 8834 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I0512 07:43:10.674667 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0512 07:43:10.674702 8834 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0512 07:43:10.674823 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0512 07:43:10.674876 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0512 07:43:10.675046 8834 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0512 07:43:10.675194 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0512 07:43:10.675249 8834 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0512 07:43:10.675293 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0512 07:43:10.675336 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:10.675388 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0512 07:43:10.675964 8834 executor.go:111] Tasks: 51 done / 93 total; 19 can run I0512 07:43:10.676117 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.externallb.example.com I0512 07:43:10.676172 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0512 07:43:10.676371 8834 iamrole.go:163] ListAttachedRolePolicies: masters.externallb.example.com I0512 07:43:10.676373 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0512 07:43:10.676426 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0512 07:43:10.676538 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0512 07:43:10.676662 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:10.676656 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0512 07:43:10.676824 8834 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0512 07:43:10.676735 8834 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] }], VpcId: "vpc-1" } I0512 07:43:10.676880 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0512 07:43:10.677107 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:10.677122 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0512 07:43:10.677272 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0512 07:43:10.677501 8834 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0512 07:43:10.677558 8834 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0512 07:43:10.677604 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0512 07:43:10.677750 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0512 07:43:10.677916 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0512 07:43:10.677994 8834 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] } I0512 07:43:10.678100 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0512 07:43:10.678193 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:10.678251 8834 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0512 07:43:10.678329 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:43:10.678397 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:43:10.678479 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0512 07:43:10.678544 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0512 07:43:10.678258 8834 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0512 07:43:10.678646 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0512 07:43:10.678711 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:10.681222 8834 executor.go:111] Tasks: 70 done / 93 total; 21 can run I0512 07:43:10.681411 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:10.681607 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0512 07:43:10.681867 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0512 07:43:10.682117 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.682222 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.externallb.example.com" }] }] } I0512 07:43:10.682428 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.682658 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0512 07:43:10.682850 8834 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0512 07:43:10.682905 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0512 07:43:10.683036 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-udp-1to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0512 07:43:10.683179 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-::/0" }] }] } I0512 07:43:10.683309 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-2382to4000-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0512 07:43:10.683444 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-4003to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0512 07:43:10.683577 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:10.683652 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:10.683713 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0512 07:43:10.683845 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-masters.externallb.example.com" }] }] } I0512 07:43:10.683978 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.684646 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.684882 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-1to2379-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0512 07:43:10.685125 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0512 07:43:10.685318 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0512 07:43:10.685527 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0512 07:43:10.685597 8834 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0512 07:43:10.685709 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0512 07:43:10.687404 8834 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0512 07:43:10.689370 8834 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0512 07:43:10.689438 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:10.689477 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:10.689544 8834 dns.go:238] Pre-creating DNS records I0512 07:43:10.689573 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:10.695838 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:10.698894 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.698997 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.740176 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:10.842235 8834 iamrole.go:91] ListRoles: { } I0512 07:43:10.843649 8834 executor.go:111] Tasks: 0 done / 93 total; 51 can run W0512 07:43:10.844767 8834 api.go:53] PageSize not implemented I0512 07:43:10.844924 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } W0512 07:43:10.845581 8834 api.go:53] PageSize not implemented I0512 07:43:10.845643 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:10.845867 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0512 07:43:10.846076 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:10.846178 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:10.843973 8834 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0512 07:43:10.846513 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } W0512 07:43:10.846601 8834 api.go:53] PageSize not implemented I0512 07:43:10.846621 8834 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0512 07:43:10.846765 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0512 07:43:10.846891 8834 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0512 07:43:10.846958 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0512 07:43:10.847048 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0512 07:43:10.847157 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0512 07:43:10.847184 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0512 07:43:10.847245 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:10.847360 8834 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0512 07:43:10.847425 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0512 07:43:10.847541 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0512 07:43:10.851879 8834 executor.go:111] Tasks: 51 done / 93 total; 19 can run I0512 07:43:10.852073 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0512 07:43:10.852157 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.externallb.example.com I0512 07:43:10.852262 8834 iamrole.go:163] ListAttachedRolePolicies: masters.externallb.example.com I0512 07:43:10.852223 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:10.852405 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0512 07:43:10.852591 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:10.852687 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0512 07:43:10.852839 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:10.852902 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0512 07:43:10.853084 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:43:10.853129 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0512 07:43:10.853292 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:10.856862 8834 executor.go:111] Tasks: 70 done / 93 total; 21 can run I0512 07:43:10.857052 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.857249 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:10.857480 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:10.857590 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.857726 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.857992 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:10.858282 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.860586 8834 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0512 07:43:10.860867 8834 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0512 07:43:10.864455 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0512 07:43:10.864539 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0512 07:43:10.864575 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0512 07:43:10.864611 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0512 07:43:10.864649 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0512 07:43:10.864686 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0512 07:43:10.864719 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0512 07:43:10.864755 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0512 07:43:10.864790 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0512 07:43:10.864825 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0512 07:43:10.864869 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0512 07:43:10.864908 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0512 07:43:10.864941 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0512 07:43:10.864974 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0512 07:43:10.865009 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0512 07:43:10.865041 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0512 07:43:10.865077 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0512 07:43:10.865112 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:43:10.865149 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0512 07:43:10.865184 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0512 07:43:10.865217 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0512 07:43:10.865249 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0512 07:43:10.865286 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0512 07:43:10.865323 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0512 07:43:10.865359 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0512 07:43:10.866951 8834 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0512 07:43:10.866979 8834 keypairs.go:95] DescribeKeyPairs: { } I0512 07:43:10.867019 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0512 07:43:10.867068 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0512 07:43:10.867125 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0512 07:43:10.867184 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0512 07:43:10.867234 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0512 07:43:10.867291 8834 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0512 07:43:10.867333 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0512 07:43:10.867384 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0512 07:43:10.867432 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0512 07:43:10.867484 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0512 07:43:10.867526 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } W0512 07:43:10.867574 8834 api.go:53] PageSize not implemented I0512 07:43:10.867594 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0512 07:43:10.867621 8834 loadbalancers.go:34] PageSize not implemented I0512 07:43:10.867638 8834 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0512 07:43:10.867661 8834 targetgroups.go:35] PageSize not implemented I0512 07:43:10.867680 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0512 07:43:10.867728 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:10.867759 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:10.867825 8834 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0512 07:43:10.867865 8834 iamrole.go:91] ListRoles: { } I0512 07:43:10.867907 8834 internetgateways.go:101] DescribeInternetGateways: { } I0512 07:43:10.867952 8834 routetable.go:53] DescribeRouteTables: { } I0512 07:43:10.867986 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:nodes.externallb.example.com ok I0512 07:43:10.868480 8834 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-group:master-us-test-1a.masters.externallb.example.com ok I0512 07:43:10.868545 8834 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0512 07:43:10.868535 8834 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0512 07:43:10.868600 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:10.868645 8834 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } iam-instance-profile:masters.externallb.example.com ok I0512 07:43:10.868693 8834 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0512 07:43:10.868747 8834 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } iam-instance-profile:nodes.externallb.example.com ok I0512 07:43:10.868641 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.externallb.example.com. ok I0512 07:43:10.868675 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0512 07:43:10.868895 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:10.868918 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0512 07:43:10.868969 8834 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0512 07:43:10.869022 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:10.869080 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0512 07:43:10.869194 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:10.869211 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0512 07:43:10.869255 8834 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-2 ok autoscaling-config:lt-1 ok I0512 07:43:10.869344 8834 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0512 07:43:10.869445 8834 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.externallb.example.com" } I0512 07:43:10.869498 8834 iamrole.go:182] ListAttachedRolePolicies: masters.externallb.example.com I0512 07:43:10.869514 8834 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0512 07:43:10.869550 8834 iamrole.go:139] DeleteRole: { RoleName: "masters.externallb.example.com" } iam-role:masters.externallb.example.com ok I0512 07:43:10.869528 8834 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0512 07:43:10.869584 8834 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.externallb.example.com" } I0512 07:43:10.869629 8834 iamrole.go:182] ListAttachedRolePolicies: nodes.externallb.example.com I0512 07:43:10.869641 8834 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0512 07:43:10.869668 8834 iamrole.go:139] DeleteRole: { RoleName: "nodes.externallb.example.com" } iam-role:nodes.externallb.example.com ok I0512 07:43:10.869729 8834 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0512 07:43:10.869808 8834 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0512 07:43:10.869883 8834 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0512 07:43:10.870163 8834 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleExternalLB (0.38s) === RUN TestLifecycleSharedSubnet I0512 07:43:10.870694 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:43:10.870745 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:43:10.870785 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:43:10.870835 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:43:10.870876 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:43:10.870908 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:43:10.870940 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:43:10.870975 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:43:10.871004 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:43:10.871061 8834 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedsubnet.example.com I0512 07:43:10.895280 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:10.899388 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.899562 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:10.937941 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:11.047877 8834 iamrole.go:91] ListRoles: { } I0512 07:43:11.049363 8834 executor.go:111] Tasks: 0 done / 79 total; 43 can run I0512 07:43:11.049714 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0512 07:43:11.049982 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:11.049966 8834 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.sharedsubnet.example.com", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "masters.sharedsubnet.example.com" }] } I0512 07:43:11.049964 8834 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0512 07:43:11.050203 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0512 07:43:11.050243 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } I0512 07:43:11.050297 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0512 07:43:11.050372 8834 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0512 07:43:11.050489 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:11.050486 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } I0512 07:43:11.050607 8834 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0512 07:43:11.050580 8834 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0512 07:43:11.050809 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0512 07:43:11.050798 8834 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0512 07:43:11.050884 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:11.051061 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0512 07:43:11.051249 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0512 07:43:11.051333 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0512 07:43:11.051364 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0512 07:43:11.052715 8834 executor.go:111] Tasks: 43 done / 79 total; 16 can run I0512 07:43:11.052836 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0512 07:43:11.052906 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0512 07:43:11.053112 8834 iamrole.go:163] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0512 07:43:11.053134 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0512 07:43:11.053220 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0512 07:43:11.053256 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0512 07:43:11.053326 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0512 07:43:11.053359 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:11.053417 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0512 07:43:11.053535 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0512 07:43:11.053727 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0512 07:43:11.053822 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0512 07:43:11.053878 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:11.053796 8834 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0512 07:43:11.053927 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0512 07:43:11.054154 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0512 07:43:11.054218 8834 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Public" } ] } I0512 07:43:11.055277 8834 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0512 07:43:11.057668 8834 executor.go:111] Tasks: 59 done / 79 total; 18 can run I0512 07:43:11.057922 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-4003to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0512 07:43:11.058183 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0512 07:43:11.058332 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0512 07:43:11.058537 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0512 07:43:11.058676 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" }] }] } I0512 07:43:11.058877 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0512 07:43:11.059096 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0512 07:43:11.059253 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0512 07:43:11.059377 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:11.059507 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:11.059631 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:11.059776 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:11.059846 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-::/0" }] }] } I0512 07:43:11.060066 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-udp-1to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0512 07:43:11.060251 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-2382to4000-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0512 07:43:11.060399 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-1to2379-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0512 07:43:11.060545 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0512 07:43:11.060678 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0512 07:43:11.062078 8834 executor.go:111] Tasks: 77 done / 79 total; 2 can run I0512 07:43:11.062372 8834 executor.go:111] Tasks: 79 done / 79 total; 0 can run I0512 07:43:11.062439 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:11.062479 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:11.062541 8834 dns.go:238] Pre-creating DNS records I0512 07:43:11.062569 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:11.068006 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:11.070768 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:11.070860 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:11.109094 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:11.212644 8834 iamrole.go:91] ListRoles: { } I0512 07:43:11.214166 8834 executor.go:111] Tasks: 0 done / 79 total; 43 can run I0512 07:43:11.214358 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0512 07:43:11.214607 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:11.214773 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0512 07:43:11.214898 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0512 07:43:11.214918 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0512 07:43:11.215011 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:11.215065 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0512 07:43:11.215306 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:11.217598 8834 executor.go:111] Tasks: 43 done / 79 total; 16 can run I0512 07:43:11.217679 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0512 07:43:11.217889 8834 iamrole.go:163] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0512 07:43:11.218745 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0512 07:43:11.217775 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0512 07:43:11.219177 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:11.219254 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0512 07:43:11.219527 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0512 07:43:11.219749 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:11.226598 8834 executor.go:111] Tasks: 59 done / 79 total; 18 can run I0512 07:43:11.226798 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:11.227140 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:11.227382 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:11.227566 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:11.229630 8834 executor.go:111] Tasks: 77 done / 79 total; 2 can run I0512 07:43:11.229854 8834 executor.go:111] Tasks: 79 done / 79 total; 0 can run I0512 07:43:11.232728 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0512 07:43:11.232793 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0512 07:43:11.232830 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0512 07:43:11.232863 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0512 07:43:11.232908 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0512 07:43:11.232931 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0512 07:43:11.232964 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0512 07:43:11.233006 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0512 07:43:11.233054 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0512 07:43:11.233111 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0512 07:43:11.233158 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0512 07:43:11.233189 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0512 07:43:11.233236 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0512 07:43:11.233278 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0512 07:43:11.233324 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0512 07:43:11.233377 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0512 07:43:11.233427 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0512 07:43:11.233455 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0512 07:43:11.233507 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0512 07:43:11.233568 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0512 07:43:11.235046 8834 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0512 07:43:11.235077 8834 keypairs.go:95] DescribeKeyPairs: { } I0512 07:43:11.235118 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0512 07:43:11.235179 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0512 07:43:11.235250 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0512 07:43:11.235316 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0512 07:43:11.235378 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0512 07:43:11.235437 8834 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0512 07:43:11.235475 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0512 07:43:11.235526 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0512 07:43:11.235577 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0512 07:43:11.235630 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0512 07:43:11.235669 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } W0512 07:43:11.235714 8834 api.go:53] PageSize not implemented I0512 07:43:11.235748 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0512 07:43:11.235773 8834 loadbalancers.go:34] PageSize not implemented I0512 07:43:11.235803 8834 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0512 07:43:11.235830 8834 targetgroups.go:35] PageSize not implemented I0512 07:43:11.235850 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0512 07:43:11.235903 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:11.235934 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:11.235983 8834 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0512 07:43:11.236019 8834 iamrole.go:91] ListRoles: { } I0512 07:43:11.236064 8834 internetgateways.go:101] DescribeInternetGateways: { } I0512 07:43:11.236105 8834 routetable.go:53] DescribeRouteTables: { } autoscaling-group:master-us-test-1a.masters.sharedsubnet.example.com ok I0512 07:43:11.236457 8834 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } autoscaling-config:lt-1 ok I0512 07:43:11.236535 8834 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } iam-instance-profile:nodes.sharedsubnet.example.com ok I0512 07:43:11.236588 8834 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0512 07:43:11.236615 8834 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0512 07:43:11.236658 8834 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } iam-instance-profile:masters.sharedsubnet.example.com ok I0512 07:43:11.236721 8834 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0512 07:43:11.236794 8834 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0512 07:43:11.236626 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.sharedsubnet.example.com. ok autoscaling-group:nodes.sharedsubnet.example.com ok autoscaling-config:lt-2 ok I0512 07:43:11.236929 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:11.237002 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } W0512 07:43:11.237159 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:11.237176 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0512 07:43:11.237200 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:11.237230 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0512 07:43:11.237308 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:11.237317 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0512 07:43:11.237378 8834 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedsubnet.example.com" } I0512 07:43:11.237413 8834 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedsubnet.example.com" } I0512 07:43:11.237488 8834 iamrole.go:182] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0512 07:43:11.237507 8834 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0512 07:43:11.237556 8834 iamrole.go:139] DeleteRole: { RoleName: "masters.sharedsubnet.example.com" } iam-role:masters.sharedsubnet.example.com ok I0512 07:43:11.237579 8834 iamrole.go:182] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0512 07:43:11.237587 8834 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0512 07:43:11.237626 8834 iamrole.go:139] DeleteRole: { RoleName: "nodes.sharedsubnet.example.com" } iam-role:nodes.sharedsubnet.example.com ok I0512 07:43:11.237927 8834 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedSubnet (0.37s) === RUN TestLifecyclePrivateSharedSubnet I0512 07:43:11.238595 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:43:11.238641 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:43:11.238681 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:43:11.238732 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:43:11.238775 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:43:11.238804 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:43:11.238836 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:43:11.238867 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:43:11.238894 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:43:11.238955 8834 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-subnet.example.com I0512 07:43:11.267082 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:11.270191 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.270283 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.270351 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.308527 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:11.414024 8834 iamrole.go:91] ListRoles: { } I0512 07:43:11.416104 8834 executor.go:111] Tasks: 0 done / 105 total; 46 can run I0512 07:43:11.416500 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:11.416568 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:11.416895 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:11.416679 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0512 07:43:11.416745 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } I0512 07:43:11.418065 8834 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0512 07:43:11.418193 8834 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0512 07:43:11.418286 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0512 07:43:11.418337 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } I0512 07:43:11.418417 8834 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.private-shared-subnet.example.com" }] } I0512 07:43:11.418374 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "private-shared-subnet.example.com" }] }] } I0512 07:43:11.418630 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0512 07:43:11.418770 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0512 07:43:11.418688 8834 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0512 07:43:11.418828 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0512 07:43:11.418894 8834 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0512 07:43:11.418938 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0512 07:43:11.419147 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } I0512 07:43:11.419267 8834 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0512 07:43:11.419265 8834 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0512 07:43:11.419534 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0512 07:43:11.419417 8834 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "bastions.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0512 07:43:11.424624 8834 executor.go:111] Tasks: 46 done / 105 total; 24 can run I0512 07:43:11.425007 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0512 07:43:11.425157 8834 iamrole.go:163] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0512 07:43:11.424910 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0512 07:43:11.425236 8834 iamrole.go:163] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0512 07:43:11.425279 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0512 07:43:11.425383 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0512 07:43:11.425413 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0512 07:43:11.425629 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0512 07:43:11.425695 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0512 07:43:11.425880 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0512 07:43:11.425908 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0512 07:43:11.426042 8834 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0512 07:43:11.426089 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0512 07:43:11.426202 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0512 07:43:11.426256 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:11.426154 8834 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0512 07:43:11.426294 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0512 07:43:11.426447 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I0512 07:43:11.426560 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0512 07:43:11.426661 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0512 07:43:11.426751 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0512 07:43:11.426791 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0512 07:43:11.426883 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0512 07:43:11.426976 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0512 07:43:11.427024 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0512 07:43:11.427062 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0512 07:43:11.427152 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0512 07:43:11.427195 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0512 07:43:11.427236 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0512 07:43:11.427330 8834 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [{ Key: "kubernetes.io/role/internal-elb", Value: "1" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" },{ Key: "SubnetType", Value: "Private" }] } I0512 07:43:11.427415 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0512 07:43:11.427518 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:11.427597 8834 tags.go:42] CreateTags { Resources: ["subnet-abcdef"], Tags: [{ Key: "kubernetes.io/role/elb", Value: "1" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" },{ Key: "SubnetType", Value: "Utility" }] } I0512 07:43:11.427774 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0512 07:43:11.427891 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0512 07:43:11.427797 8834 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0512 07:43:11.430042 8834 executor.go:111] Tasks: 70 done / 105 total; 31 can run I0512 07:43:11.430256 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } W0512 07:43:11.430425 8834 api.go:53] PageSize not implemented I0512 07:43:11.430435 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } W0512 07:43:11.430520 8834 api.go:53] PageSize not implemented I0512 07:43:11.430564 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0512 07:43:11.430665 8834 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"], Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0512 07:43:11.430544 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0512 07:43:11.430738 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0512 07:43:11.430791 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.430886 8834 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0512 07:43:11.430991 8834 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0512 07:43:11.431117 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0512 07:43:11.431178 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0512 07:43:11.431262 8834 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "api.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] } I0512 07:43:11.431364 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0512 07:43:11.431415 8834 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0512 07:43:11.431371 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0512 07:43:11.431466 8834 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0512 07:43:11.431582 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.431738 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0512 07:43:11.431902 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0512 07:43:11.432043 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0512 07:43:11.432254 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.432414 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0512 07:43:11.432617 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-ingress-tcp-22to22-bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0512 07:43:11.432793 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0512 07:43:11.432944 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-4003to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0512 07:43:11.433092 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0512 07:43:11.433197 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0512 07:43:11.433342 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0512 07:43:11.433493 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0512 07:43:11.433702 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0512 07:43:11.433860 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.433963 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.434028 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0512 07:43:11.434206 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } I0512 07:43:11.434314 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I0512 07:43:11.434441 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0512 07:43:11.434634 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-2382to4000-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0512 07:43:11.434820 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-1to2379-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0512 07:43:11.434967 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-subnet.example.com" }] }] } I0512 07:43:11.435085 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-udp-1to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0512 07:43:11.435319 8834 executor.go:111] Tasks: 101 done / 105 total; 4 can run I0512 07:43:11.435523 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0512 07:43:11.435689 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0512 07:43:11.435824 8834 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0512 07:43:11.435906 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:11.435942 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:11.435996 8834 dns.go:238] Pre-creating DNS records I0512 07:43:11.436021 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:11.442358 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:11.445334 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.445428 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.445496 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.481717 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:11.591994 8834 iamrole.go:91] ListRoles: { } I0512 07:43:11.594148 8834 executor.go:111] Tasks: 0 done / 105 total; 46 can run I0512 07:43:11.594390 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] }] } I0512 07:43:11.594830 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:11.595020 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:11.594975 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0512 07:43:11.595288 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:11.596855 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0512 07:43:11.597003 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0512 07:43:11.597092 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0512 07:43:11.599824 8834 executor.go:111] Tasks: 46 done / 105 total; 24 can run I0512 07:43:11.600080 8834 iamrole.go:163] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0512 07:43:11.600202 8834 iamrole.go:163] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0512 07:43:11.600243 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0512 07:43:11.600342 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0512 07:43:11.601230 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0512 07:43:11.601752 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I0512 07:43:11.601982 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0512 07:43:11.602046 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0512 07:43:11.602218 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0512 07:43:11.602273 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0512 07:43:11.602378 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0512 07:43:11.603359 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:11.603451 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:11.603564 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0512 07:43:11.603756 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0512 07:43:11.603958 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0512 07:43:11.608173 8834 executor.go:111] Tasks: 70 done / 105 total; 31 can run W0512 07:43:11.608310 8834 api.go:53] PageSize not implemented I0512 07:43:11.608345 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0512 07:43:11.608455 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0512 07:43:11.608496 8834 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0512 07:43:11.608477 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } W0512 07:43:11.608591 8834 api.go:53] PageSize not implemented I0512 07:43:11.608609 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I0512 07:43:11.608688 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0512 07:43:11.608740 8834 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0512 07:43:11.608718 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.608918 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.609103 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.609354 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.609487 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.611822 8834 executor.go:111] Tasks: 101 done / 105 total; 4 can run W0512 07:43:11.611931 8834 api.go:53] PageSize not implemented I0512 07:43:11.611954 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0512 07:43:11.612019 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } W0512 07:43:11.612126 8834 api.go:53] PageSize not implemented I0512 07:43:11.612147 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0512 07:43:11.612124 8834 dnsname.go:121] AliasTarget for "api.private-shared-subnet.example.com" is "api-private-shared-subnet-n2f8ak.elb.cloudmock.com" I0512 07:43:11.612311 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0512 07:43:11.612429 8834 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0512 07:43:11.616172 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0512 07:43:11.616236 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0512 07:43:11.616274 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0512 07:43:11.616309 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0512 07:43:11.616348 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0512 07:43:11.616381 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0512 07:43:11.616414 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0512 07:43:11.616453 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0512 07:43:11.616490 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0512 07:43:11.616536 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0512 07:43:11.616576 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0512 07:43:11.616610 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0512 07:43:11.616643 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0512 07:43:11.616675 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0512 07:43:11.616711 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0512 07:43:11.616746 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0512 07:43:11.616780 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0512 07:43:11.616817 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0512 07:43:11.616852 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0512 07:43:11.616887 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0512 07:43:11.616924 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0512 07:43:11.616958 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0512 07:43:11.616994 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0512 07:43:11.617031 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0512 07:43:11.618701 8834 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0512 07:43:11.618730 8834 keypairs.go:95] DescribeKeyPairs: { } I0512 07:43:11.618771 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0512 07:43:11.618835 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0512 07:43:11.618918 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0512 07:43:11.618980 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0512 07:43:11.619024 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0512 07:43:11.619069 8834 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0512 07:43:11.619114 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0512 07:43:11.619151 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0512 07:43:11.619190 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0512 07:43:11.619234 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0512 07:43:11.619271 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } W0512 07:43:11.619311 8834 api.go:53] PageSize not implemented I0512 07:43:11.619330 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0512 07:43:11.619383 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0512 07:43:11.619411 8834 loadbalancers.go:34] PageSize not implemented I0512 07:43:11.619430 8834 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0512 07:43:11.619453 8834 targetgroups.go:35] PageSize not implemented I0512 07:43:11.619472 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0512 07:43:11.619529 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:11.619559 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:11.619609 8834 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0512 07:43:11.619647 8834 iamrole.go:91] ListRoles: { } I0512 07:43:11.619690 8834 internetgateways.go:101] DescribeInternetGateways: { } I0512 07:43:11.619733 8834 routetable.go:53] DescribeRouteTables: { } I0512 07:43:11.620241 8834 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } autoscaling-config:lt-2 ok I0512 07:43:11.620306 8834 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } iam-instance-profile:masters.private-shared-subnet.example.com ok autoscaling-config:lt-1 ok I0512 07:43:11.620340 8834 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } load-balancer:bastion-private-shared-su-5ol32q ok I0512 07:43:11.620360 8834 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } autoscaling-group:master-us-test-1a.masters.private-shared-subnet.example.com ok autoscaling-group:bastion.private-shared-subnet.example.com ok I0512 07:43:11.620449 8834 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-group:nodes.private-shared-subnet.example.com ok I0512 07:43:11.620518 8834 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } I0512 07:43:11.620523 8834 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } volume:vol-1 ok I0512 07:43:11.620442 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.private-shared-subnet.example.com. ok I0512 07:43:11.620603 8834 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } iam-instance-profile:nodes.private-shared-subnet.example.com ok I0512 07:43:11.620618 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0512 07:43:11.620632 8834 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } keypair:key-1 ok I0512 07:43:11.620667 8834 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0512 07:43:11.620715 8834 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } load-balancer:api-private-shared-subnet-n2f8ak ok iam-instance-profile:bastions.private-shared-subnet.example.com ok I0512 07:43:11.620695 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0512 07:43:11.620954 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:11.620987 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0512 07:43:11.621028 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:11.621113 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W0512 07:43:11.621247 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:11.621270 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-3 ok I0512 07:43:11.621324 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:11.621397 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0512 07:43:11.621530 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:11.621551 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0512 07:43:11.621656 8834 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-subnet.example.com" } I0512 07:43:11.621725 8834 iamrole.go:182] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0512 07:43:11.621743 8834 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0512 07:43:11.621718 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0512 07:43:11.621781 8834 iamrole.go:139] DeleteRole: { RoleName: "nodes.private-shared-subnet.example.com" } iam-role:nodes.private-shared-subnet.example.com ok I0512 07:43:11.621803 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0512 07:43:11.621905 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:11.621920 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0512 07:43:11.621834 8834 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-subnet.example.com" } I0512 07:43:11.621985 8834 iamrole.go:182] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0512 07:43:11.621962 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0512 07:43:11.622011 8834 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-subnet.example.com" } I0512 07:43:11.622067 8834 iamrole.go:182] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0512 07:43:11.622091 8834 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0512 07:43:11.622163 8834 iamrole.go:139] DeleteRole: { RoleName: "bastions.private-shared-subnet.example.com" } I0512 07:43:11.622036 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } iam-role:bastions.private-shared-subnet.example.com ok W0512 07:43:11.622202 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:11.622206 8834 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0512 07:43:11.622221 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0512 07:43:11.622241 8834 iamrole.go:139] DeleteRole: { RoleName: "masters.private-shared-subnet.example.com" } iam-role:masters.private-shared-subnet.example.com ok I0512 07:43:11.622556 8834 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedSubnet (0.38s) === RUN TestLifecyclePrivateSharedIP I0512 07:43:11.623085 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:43:11.623133 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:43:11.623174 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:43:11.623221 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:43:11.623260 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:43:11.623291 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:43:11.623320 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:43:11.623354 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:43:11.623379 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:43:11.623436 8834 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-ip.example.com I0512 07:43:11.651324 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:11.654226 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.654315 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.654378 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.691992 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:11.798739 8834 iamrole.go:91] ListRoles: { } I0512 07:43:11.800920 8834 executor.go:111] Tasks: 0 done / 114 total; 46 can run I0512 07:43:11.801160 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:11.801438 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0512 07:43:11.801628 8834 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0512 07:43:11.801660 8834 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0512 07:43:11.801819 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0512 07:43:11.801873 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } I0512 07:43:11.801896 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] }] } I0512 07:43:11.801955 8834 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0512 07:43:11.802000 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:11.802101 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:11.802146 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } I0512 07:43:11.802208 8834 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0512 07:43:11.802398 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0512 07:43:11.802313 8834 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0512 07:43:11.802467 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0512 07:43:11.802476 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } I0512 07:43:11.802566 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0512 07:43:11.802598 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0512 07:43:11.802585 8834 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0512 07:43:11.802635 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0512 07:43:11.802721 8834 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0512 07:43:11.803065 8834 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "bastions.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0512 07:43:11.805666 8834 executor.go:111] Tasks: 46 done / 114 total; 26 can run I0512 07:43:11.805877 8834 iamrole.go:163] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0512 07:43:11.805921 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0512 07:43:11.806100 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0512 07:43:11.806307 8834 iamrole.go:163] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0512 07:43:11.806245 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0512 07:43:11.806499 8834 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0512 07:43:11.806506 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0512 07:43:11.806750 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0512 07:43:11.806756 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0512 07:43:11.806972 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0512 07:43:11.807032 8834 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-12345678" } I0512 07:43:11.807002 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0512 07:43:11.807245 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0512 07:43:11.807367 8834 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0512 07:43:11.807376 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0512 07:43:11.807589 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0512 07:43:11.807781 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0512 07:43:11.807995 8834 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "private-shared-ip.example.com" } ] }], VpcId: "vpc-12345678" } I0512 07:43:11.808057 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I0512 07:43:11.808890 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0512 07:43:11.809047 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0512 07:43:11.809150 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I0512 07:43:11.808450 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" } ] }], VpcId: "vpc-12345678" } I0512 07:43:11.809299 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0512 07:43:11.809459 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0512 07:43:11.810281 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0512 07:43:11.809557 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], VpcId: "vpc-12345678" } I0512 07:43:11.810489 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0512 07:43:11.810648 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:43:11.810790 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:43:11.810899 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0512 07:43:11.810986 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0512 07:43:11.810734 8834 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0512 07:43:11.811275 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0512 07:43:11.811390 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0512 07:43:11.811544 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0512 07:43:11.811614 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0512 07:43:11.811680 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0512 07:43:11.811753 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0512 07:43:11.811820 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0512 07:43:11.811886 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0512 07:43:11.811958 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0512 07:43:11.812026 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0512 07:43:11.812121 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0512 07:43:11.812187 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:11.812265 8834 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] } I0512 07:43:11.812366 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:11.812410 8834 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "private-shared-ip.example.com" } ] } I0512 07:43:11.815310 8834 executor.go:111] Tasks: 72 done / 114 total; 36 can run I0512 07:43:11.815549 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W0512 07:43:11.815670 8834 api.go:53] PageSize not implemented I0512 07:43:11.815699 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0512 07:43:11.815824 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0512 07:43:11.815858 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:11.815941 8834 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"], Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0512 07:43:11.816017 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0512 07:43:11.816031 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0512 07:43:11.816051 8834 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0512 07:43:11.816120 8834 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } W0512 07:43:11.816206 8834 api.go:53] PageSize not implemented I0512 07:43:11.816087 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0512 07:43:11.816224 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0512 07:43:11.816398 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0512 07:43:11.816435 8834 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"], Tags: [{ Key: "Name", Value: "api.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0512 07:43:11.816491 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0512 07:43:11.816398 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-ingress-tcp-22to22-bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0512 07:43:11.816520 8834 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0512 07:43:11.816566 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0512 07:43:11.816566 8834 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0512 07:43:11.816700 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-4003to65535-masters.private-shared-ip.example.com" }] }] } I0512 07:43:11.816960 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.817132 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:11.817235 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:11.817318 8834 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0512 07:43:11.817433 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:11.817541 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-udp-1to65535-masters.private-shared-ip.example.com" }] }] } I0512 07:43:11.817768 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0512 07:43:11.817961 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-1to2379-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0512 07:43:11.818144 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I0512 07:43:11.818304 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0512 07:43:11.818482 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0512 07:43:11.818661 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.818804 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0512 07:43:11.818963 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-2382to4000-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0512 07:43:11.819150 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-ip.example.com" }] }] } I0512 07:43:11.819319 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0512 07:43:11.819490 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0512 07:43:11.819660 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0512 07:43:11.819726 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0512 07:43:11.819785 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0512 07:43:11.819924 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0512 07:43:11.820080 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0512 07:43:11.820250 8834 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-2" } I0512 07:43:11.820308 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0512 07:43:11.820473 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0512 07:43:11.820611 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I0512 07:43:11.820822 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0512 07:43:11.820978 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.821092 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.821168 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.821300 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.821437 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0512 07:43:11.821675 8834 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-2" } I0512 07:43:11.821852 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0512 07:43:11.821956 8834 tags.go:42] CreateTags { Resources: ["eipalloc-12345678"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "shared" }] } I0512 07:43:11.822042 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0512 07:43:11.823692 8834 executor.go:111] Tasks: 108 done / 114 total; 5 can run I0512 07:43:11.823995 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:11.824100 8834 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-2"] }] } I0512 07:43:11.824005 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0512 07:43:11.824237 8834 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-12345678", SubnetId: "subnet-2", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0512 07:43:11.824263 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0512 07:43:11.824352 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0512 07:43:11.824402 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0512 07:43:11.824446 8834 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0512 07:43:11.824540 8834 executor.go:111] Tasks: 113 done / 114 total; 1 can run I0512 07:43:11.824602 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:11.824729 8834 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I0512 07:43:11.824798 8834 executor.go:111] Tasks: 114 done / 114 total; 0 can run I0512 07:43:11.824852 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:11.824885 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:11.824941 8834 dns.go:238] Pre-creating DNS records I0512 07:43:11.824969 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:11.831441 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0512 07:43:11.834453 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.834567 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.834634 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.872155 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:11.978128 8834 iamrole.go:91] ListRoles: { } I0512 07:43:11.980324 8834 executor.go:111] Tasks: 0 done / 114 total; 46 can run I0512 07:43:11.980661 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0512 07:43:11.980934 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:11.981057 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:11.981169 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:11.981252 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0512 07:43:11.981439 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0512 07:43:11.981516 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0512 07:43:11.981543 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0512 07:43:11.983538 8834 executor.go:111] Tasks: 46 done / 114 total; 26 can run I0512 07:43:11.983694 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I0512 07:43:11.983886 8834 iamrole.go:163] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0512 07:43:11.983959 8834 iamrole.go:163] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0512 07:43:11.983918 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0512 07:43:11.984029 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0512 07:43:11.984045 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0512 07:43:11.984299 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0512 07:43:11.984396 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] }] } I0512 07:43:11.984647 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0512 07:43:11.984717 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0512 07:43:11.984988 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:11.985052 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0512 07:43:11.985214 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0512 07:43:11.985364 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0512 07:43:11.986176 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0512 07:43:11.986465 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:43:11.986601 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0512 07:43:11.986794 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:11.986939 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0512 07:43:11.987096 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0512 07:43:11.993002 8834 executor.go:111] Tasks: 72 done / 114 total; 36 can run I0512 07:43:11.993179 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0512 07:43:11.993310 8834 api.go:53] PageSize not implemented I0512 07:43:11.993348 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I0512 07:43:11.993479 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0512 07:43:11.993391 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.993525 8834 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0512 07:43:11.993541 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0512 07:43:11.993621 8834 api.go:53] PageSize not implemented I0512 07:43:11.993700 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I0512 07:43:11.993788 8834 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0512 07:43:11.993876 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0512 07:43:11.993831 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0512 07:43:11.993973 8834 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0512 07:43:11.993987 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.994135 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.994461 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.994622 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:11.994723 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0512 07:43:11.994820 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.994869 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0512 07:43:11.997024 8834 executor.go:111] Tasks: 108 done / 114 total; 5 can run I0512 07:43:11.997155 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } W0512 07:43:11.997248 8834 api.go:53] PageSize not implemented I0512 07:43:11.997255 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:11.997273 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I0512 07:43:11.997244 8834 dnsname.go:121] AliasTarget for "api.private-shared-ip.example.com" is "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com" I0512 07:43:11.997354 8834 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0512 07:43:11.997408 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } W0512 07:43:11.997524 8834 api.go:53] PageSize not implemented I0512 07:43:11.997559 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I0512 07:43:11.997796 8834 executor.go:111] Tasks: 113 done / 114 total; 1 can run I0512 07:43:11.997929 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:11.998068 8834 executor.go:111] Tasks: 114 done / 114 total; 0 can run I0512 07:43:12.001839 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0512 07:43:12.001969 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0512 07:43:12.002049 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0512 07:43:12.002114 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0512 07:43:12.002182 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0512 07:43:12.002246 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0512 07:43:12.002307 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0512 07:43:12.002374 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0512 07:43:12.002440 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0512 07:43:12.002506 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0512 07:43:12.002578 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0512 07:43:12.002639 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0512 07:43:12.002695 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0512 07:43:12.002751 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0512 07:43:12.002820 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0512 07:43:12.002875 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0512 07:43:12.002943 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0512 07:43:12.003005 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0512 07:43:12.003068 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0512 07:43:12.003129 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0512 07:43:12.003191 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0512 07:43:12.003252 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:43:12.003316 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0512 07:43:12.003370 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0512 07:43:12.003444 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0512 07:43:12.003500 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0512 07:43:12.003561 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0512 07:43:12.003627 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0512 07:43:12.003689 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0512 07:43:12.005535 8834 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0512 07:43:12.005575 8834 keypairs.go:95] DescribeKeyPairs: { } I0512 07:43:12.005631 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0512 07:43:12.005725 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0512 07:43:12.005833 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0512 07:43:12.005919 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0512 07:43:12.005979 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0512 07:43:12.006045 8834 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0512 07:43:12.006107 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0512 07:43:12.006185 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0512 07:43:12.006269 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0512 07:43:12.006346 8834 routetable.go:53] DescribeRouteTables: { } I0512 07:43:12.006391 8834 natgateway.go:117] DescribeNatGateways: { } I0512 07:43:12.006461 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0512 07:43:12.006514 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } W0512 07:43:12.006582 8834 api.go:53] PageSize not implemented I0512 07:43:12.006611 8834 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I0512 07:43:12.006693 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0512 07:43:12.006733 8834 loadbalancers.go:34] PageSize not implemented I0512 07:43:12.006759 8834 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0512 07:43:12.006798 8834 targetgroups.go:35] PageSize not implemented I0512 07:43:12.006825 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0512 07:43:12.006897 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:12.006941 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:12.007007 8834 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0512 07:43:12.007065 8834 iamrole.go:91] ListRoles: { } I0512 07:43:12.007135 8834 internetgateways.go:101] DescribeInternetGateways: { } I0512 07:43:12.007203 8834 routetable.go:53] DescribeRouteTables: { } I0512 07:43:12.007260 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I0512 07:43:12.007316 8834 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0512 07:43:12.007374 8834 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } autoscaling-group:nodes.private-shared-ip.example.com ok I0512 07:43:12.008457 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:master-us-test-1a.masters.private-shared-ip.example.com ok I0512 07:43:12.008511 8834 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0512 07:43:12.008567 8834 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } load-balancer:bastion-private-shared-ip-eepmph ok I0512 07:43:12.008617 8834 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } autoscaling-group:bastion.private-shared-ip.example.com ok I0512 07:43:12.008581 8834 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0512 07:43:12.008702 8834 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } iam-instance-profile:nodes.private-shared-ip.example.com ok autoscaling-config:lt-2 ok I0512 07:43:12.008699 8834 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } autoscaling-config:lt-3 ok load-balancer:api-private-shared-ip-exa-ohatqj ok I0512 07:43:12.008745 8834 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } iam-instance-profile:masters.private-shared-ip.example.com ok I0512 07:43:12.008766 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:12.008794 8834 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0512 07:43:12.008859 8834 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } iam-instance-profile:bastions.private-shared-ip.example.com ok I0512 07:43:12.008735 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.private-shared-ip.example.com. ok I0512 07:43:12.008839 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W0512 07:43:12.009074 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:12.009097 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0512 07:43:12.009132 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0512 07:43:12.009237 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:12.009257 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0512 07:43:12.009300 8834 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0512 07:43:12.009361 8834 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0512 07:43:12.009417 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0512 07:43:12.009469 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } W0512 07:43:12.009547 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:12.009563 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok autoscaling-config:lt-1 ok I0512 07:43:12.009614 8834 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0512 07:43:12.009721 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0512 07:43:12.009758 8834 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-ip.example.com" } I0512 07:43:12.009772 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0512 07:43:12.009851 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:12.009864 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0512 07:43:12.009833 8834 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-ip.example.com" } I0512 07:43:12.009922 8834 iamrole.go:182] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0512 07:43:12.009898 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0512 07:43:12.009955 8834 iamrole.go:182] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0512 07:43:12.010039 8834 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0512 07:43:12.010093 8834 iamrole.go:139] DeleteRole: { RoleName: "nodes.private-shared-ip.example.com" } iam-role:nodes.private-shared-ip.example.com ok I0512 07:43:12.010012 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } W0512 07:43:12.010141 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:12.010132 8834 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-ip.example.com" } I0512 07:43:12.010163 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0512 07:43:12.010187 8834 iamrole.go:182] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0512 07:43:12.010201 8834 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0512 07:43:12.010231 8834 iamrole.go:139] DeleteRole: { RoleName: "bastions.private-shared-ip.example.com" } iam-role:bastions.private-shared-ip.example.com ok I0512 07:43:12.010229 8834 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0512 07:43:12.010265 8834 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0512 07:43:12.010332 8834 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0512 07:43:12.010341 8834 iamrole.go:139] DeleteRole: { RoleName: "masters.private-shared-ip.example.com" } iam-role:masters.private-shared-ip.example.com ok I0512 07:43:12.010450 8834 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0512 07:43:12.010536 8834 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0512 07:43:12.010970 8834 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedIP (0.39s) === RUN TestLifecycleNodeTerminationHandlerQueueProcessor I0512 07:43:12.011723 8834 internetgateways.go:66] CreateInternetGateway: { } I0512 07:43:12.011771 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:43:12.011811 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:43:12.011865 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:43:12.011910 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:43:12.011940 8834 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:43:12.011970 8834 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:43:12.012001 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:43:12.012027 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:43:12.012084 8834 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster nthsqsresources.longclustername.example.com I0512 07:43:12.037072 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.13 ********************************************************************************* I0512 07:43:12.039997 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:12.040088 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:12.080487 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:12.207136 8834 iamrole.go:91] ListRoles: { } I0512 07:43:12.208762 8834 executor.go:111] Tasks: 0 done / 98 total; 46 can run I0512 07:43:12.209135 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:12.209151 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:12.209224 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:12.209402 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0512 07:43:12.210257 8834 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0512 07:43:12.210401 8834 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0512 07:43:12.209276 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } I0512 07:43:12.210440 8834 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0512 07:43:12.210729 8834 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" }] } I0512 07:43:12.210829 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0512 07:43:12.210972 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:12.211024 8834 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.nthsqsresources.longclustername.example.com", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" }] } I0512 07:43:12.211311 8834 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I0512 07:43:12.211447 8834 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0512 07:43:12.211895 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0512 07:43:12.212381 8834 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0512 07:43:12.212797 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0512 07:43:12.212881 8834 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } I0512 07:43:12.212911 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0512 07:43:12.212981 8834 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I0512 07:43:12.213160 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0512 07:43:12.213441 8834 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.nthsqsresources.longclustername.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0512 07:43:12.215493 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0512 07:43:12.215591 8834 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0512 07:43:12.215784 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0512 07:43:12.216667 8834 executor.go:111] Tasks: 46 done / 98 total; 23 can run I0512 07:43:12.216809 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0512 07:43:12.216923 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0512 07:43:12.217009 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0512 07:43:12.217014 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:12.217056 8834 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0512 07:43:12.217190 8834 iamrole.go:163] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0512 07:43:12.217132 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] }] } I0512 07:43:12.217241 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0512 07:43:12.217334 8834 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0512 07:43:12.217336 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I0512 07:43:12.217511 8834 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0512 07:43:12.217639 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }], VpcId: "vpc-1" } I0512 07:43:12.217777 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0512 07:43:12.217871 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:12.217814 8834 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0512 07:43:12.217926 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0512 07:43:12.218042 8834 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0512 07:43:12.218154 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0512 07:43:12.218300 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I0512 07:43:12.218479 8834 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0512 07:43:12.218555 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:12.218665 8834 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0512 07:43:12.218778 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0512 07:43:12.218847 8834 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0512 07:43:12.218908 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:43:12.218971 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:43:12.219031 8834 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0512 07:43:12.219157 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0512 07:43:12.219219 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:12.218958 8834 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\",\n \"sqs:DeleteMessage\",\n \"sqs:ReceiveMessage\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:CompleteLifecycleAction\",\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0512 07:43:12.222378 8834 executor.go:111] Tasks: 69 done / 98 total; 25 can run I0512 07:43:12.222624 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0512 07:43:12.222887 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-4003to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0512 07:43:12.223058 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:12.223209 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0512 07:43:12.223254 8834 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0512 07:43:12.223368 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0512 07:43:12.223516 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0512 07:43:12.223743 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-masters.nthsqsresources.longclustername.example.com" }] }] } I0512 07:43:12.223894 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:12.223975 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:12.224057 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0512 07:43:12.224200 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0512 07:43:12.224358 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0512 07:43:12.224522 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0512 07:43:12.224655 8834 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0512 07:43:12.224705 8834 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0512 07:43:12.224849 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.nthsqsresources.longclustername.example.com" }] }] } I0512 07:43:12.225008 8834 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0512 07:43:12.225052 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-1to2379-masters.nthsqsresources.longclustername.example.com" }] }] } I0512 07:43:12.225204 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:12.225312 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:12.225408 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-2382to4000-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0512 07:43:12.225638 8834 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-udp-1to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0512 07:43:12.225820 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:12.226000 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:12.227900 8834 executor.go:111] Tasks: 94 done / 98 total; 2 can run I0512 07:43:12.228163 8834 executor.go:111] Tasks: 96 done / 98 total; 2 can run I0512 07:43:12.228366 8834 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0512 07:43:12.228426 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:12.228473 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:12.228537 8834 dns.go:238] Pre-creating DNS records I0512 07:43:12.228567 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:12.234458 8834 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.13 ********************************************************************************* I0512 07:43:12.237378 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:12.237471 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:12.275315 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:12.403162 8834 iamrole.go:91] ListRoles: { } I0512 07:43:12.404858 8834 executor.go:111] Tasks: 0 done / 98 total; 46 can run I0512 07:43:12.405166 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:12.405276 8834 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0512 07:43:12.405384 8834 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0512 07:43:12.405425 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0512 07:43:12.405643 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0512 07:43:12.405889 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0512 07:43:12.406057 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0512 07:43:12.406158 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0512 07:43:12.406189 8834 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0512 07:43:12.406237 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:12.409160 8834 executor.go:111] Tasks: 46 done / 98 total; 23 can run I0512 07:43:12.409388 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:12.409532 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] }] } I0512 07:43:12.409703 8834 iamrole.go:163] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0512 07:43:12.409870 8834 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:43:12.410026 8834 iamrole.go:163] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0512 07:43:12.410248 8834 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:43:12.410411 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0512 07:43:12.410717 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I0512 07:43:12.411122 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I0512 07:43:12.411506 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0512 07:43:12.412033 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:12.412147 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:12.417642 8834 executor.go:111] Tasks: 69 done / 98 total; 25 can run I0512 07:43:12.417923 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:12.418141 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:12.418455 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:12.418612 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:12.418784 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0512 07:43:12.418968 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:12.419147 8834 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0512 07:43:12.422056 8834 executor.go:111] Tasks: 94 done / 98 total; 2 can run I0512 07:43:12.422306 8834 executor.go:111] Tasks: 96 done / 98 total; 2 can run I0512 07:43:12.422513 8834 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0512 07:43:12.425468 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0512 07:43:12.425533 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0512 07:43:12.425571 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0512 07:43:12.425604 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0512 07:43:12.425643 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0512 07:43:12.425678 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0512 07:43:12.425709 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0512 07:43:12.425743 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0512 07:43:12.425779 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0512 07:43:12.425815 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0512 07:43:12.425853 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0512 07:43:12.425902 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0512 07:43:12.425934 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0512 07:43:12.425967 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0512 07:43:12.426004 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0512 07:43:12.426038 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0512 07:43:12.426071 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0512 07:43:12.426107 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:43:12.426146 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0512 07:43:12.426179 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0512 07:43:12.426211 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0512 07:43:12.426243 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0512 07:43:12.426280 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0512 07:43:12.426317 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0512 07:43:12.426352 8834 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0512 07:43:12.427883 8834 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0512 07:43:12.427913 8834 keypairs.go:95] DescribeKeyPairs: { } I0512 07:43:12.427953 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0512 07:43:12.428006 8834 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I0512 07:43:12.428068 8834 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0512 07:43:12.428128 8834 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0512 07:43:12.428178 8834 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0512 07:43:12.428236 8834 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0512 07:43:12.428281 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0512 07:43:12.428323 8834 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I0512 07:43:12.428373 8834 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0512 07:43:12.428425 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0512 07:43:12.428467 8834 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } W0512 07:43:12.428519 8834 api.go:53] PageSize not implemented I0512 07:43:12.428542 8834 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0512 07:43:12.428570 8834 loadbalancers.go:34] PageSize not implemented I0512 07:43:12.428589 8834 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0512 07:43:12.428614 8834 targetgroups.go:35] PageSize not implemented I0512 07:43:12.428633 8834 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0512 07:43:12.428706 8834 zones.go:93] ListHostedZonesPages { } I0512 07:43:12.428739 8834 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0512 07:43:12.428788 8834 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0512 07:43:12.428826 8834 iamrole.go:91] ListRoles: { } I0512 07:43:12.428872 8834 internetgateways.go:101] DescribeInternetGateways: { } I0512 07:43:12.428915 8834 routetable.go:53] DescribeRouteTables: { } I0512 07:43:12.428949 8834 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:master-us-test-1a.masters.nthsqsresources.longclustername.example.com ok I0512 07:43:12.429432 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0512 07:43:12.429491 8834 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } autoscaling-group:nodes.nthsqsresources.longclustername.example.com ok I0512 07:43:12.429582 8834 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } iam-instance-profile:masters.nthsqsresources.longclustername.example.com ok I0512 07:43:12.429645 8834 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0512 07:43:12.429495 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } I0512 07:43:12.429694 8834 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } iam-instance-profile:nodes.nthsqsresources.longclustername.example.com ok W0512 07:43:12.429711 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:12.429728 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0512 07:43:12.429605 8834 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.nthsqsresources.longclustername.example.com. ok I0512 07:43:12.429779 8834 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0512 07:43:12.429868 8834 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-2 ok I0512 07:43:12.429948 8834 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0512 07:43:12.430010 8834 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0512 07:43:12.430146 8834 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0512 07:43:12.430162 8834 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0512 07:43:12.430206 8834 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-1 ok I0512 07:43:12.430278 8834 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0512 07:43:12.430334 8834 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0512 07:43:12.430426 8834 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0512 07:43:12.430475 8834 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.nthsqsresources.longclustername.example.com" } I0512 07:43:12.430565 8834 iamrole.go:182] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0512 07:43:12.430584 8834 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0512 07:43:12.430621 8834 iamrole.go:139] DeleteRole: { RoleName: "masters.nthsqsresources.longclustername.example.com" } iam-role:masters.nthsqsresources.longclustername.example.com ok I0512 07:43:12.430662 8834 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0512 07:43:12.430708 8834 iamrole.go:182] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0512 07:43:12.430720 8834 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0512 07:43:12.430750 8834 iamrole.go:139] DeleteRole: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } iam-role:nodes.nthsqsresources.longclustername.example.com ok I0512 07:43:12.430826 8834 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0512 07:43:12.430908 8834 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0512 07:43:12.430983 8834 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0512 07:43:12.431276 8834 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleNodeTerminationHandlerQueueProcessor (0.42s) === RUN TestGetFilters --- PASS: TestGetFilters (0.00s) === RUN TestGetInstanceSelectorOpts --- PASS: TestGetInstanceSelectorOpts (0.00s) === RUN TestValidateAllPrivateOrPublicSubnets --- PASS: TestValidateAllPrivateOrPublicSubnets (0.00s) === RUN TestValidateUserSubnetsWithClusterSubnets --- PASS: TestValidateUserSubnetsWithClusterSubnets (0.00s) === RUN TestCreateInstanceGroup --- PASS: TestCreateInstanceGroup (0.00s) === RUN TestDecorateWithInstanceGroupSpecs --- PASS: TestDecorateWithInstanceGroupSpecs (0.00s) === RUN TestDecorateWithMixedInstancesPolicy --- PASS: TestDecorateWithMixedInstancesPolicy (0.00s) === RUN TestDecorateWithClusterAutoscalerLabels --- PASS: TestDecorateWithClusterAutoscalerLabels (0.00s) === RUN TestNewTemplateContext --- PASS: TestNewTemplateContext (0.00s) PASS ok k8s.io/kops/cmd/kops 53.189s ? k8s.io/kops/cmd/kops/util [no test files] ? k8s.io/kops/cmd/kops-controller [no test files] ? k8s.io/kops/cmd/kops-controller/controllers [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/config [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/server [no test files] === RUN TestBuildProxyRequest === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/livez === RUN TestBuildProxyRequest/http://127.0.0.1:3990/healthz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ready === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz? === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude --- PASS: TestBuildProxyRequest (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/livez (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/healthz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ready (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz? (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude (0.00s) PASS ok k8s.io/kops/cmd/kube-apiserver-healthcheck 0.028s ? k8s.io/kops/cmd/nodeup [no test files] ? k8s.io/kops/dns-controller/cmd/dns-controller [no test files] === RUN TestAliasForNodesInRole --- PASS: TestAliasForNodesInRole (0.00s) === RUN TestEnsureDotSuffix --- PASS: TestEnsureDotSuffix (0.00s) === RUN TestParseZoneSpec --- PASS: TestParseZoneSpec (0.00s) === RUN TestParseZoneRules I0512 07:41:55.247738 3145 zonespec.go:75] No rules specified, will permit management of all zones --- PASS: TestParseZoneRules (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/dns 0.047s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/util 0.038s === RUN TestPodController I0512 07:41:58.852667 4842 pod.go:59] starting pod controller I0512 07:41:58.852930 4842 stoppable.go:62] shutting down controller I0512 07:41:58.853041 4842 pod.go:107] Got stop signal I0512 07:41:58.853158 4842 pod.go:65] shutting down pod controller --- PASS: TestPodController (0.01s) PASS ok k8s.io/kops/dns-controller/pkg/watchers 0.055s === RUN TestEquivalent --- PASS: TestEquivalent (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider 0.019s Parsing flags. Getting new test interface. Running tests... === RUN TestZonesList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000193d50} route53_test.go:93: Got zone list: [0xc000193d80] route53_test.go:98: Got at least 1 zone in list:&{0xc0003a0480 0xc00019c080} --- PASS: TestZonesList (0.00s) === RUN TestZonesID route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000193e50} route53_test.go:93: Got zone list: [0xc000193e80] route53_test.go:98: Got at least 1 zone in list:&{0xc0003a0480 0xc00019c090} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess route53_test.go:79: Got zones {0xc000193f20} route53_test.go:170: Successfully added managed DNS zone: &{0xc0003a0660 0xc00019c0a8} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003bc080} route53_test.go:93: Got zone list: [0xc0003bc0b0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003a0480 0xc00019c0c0} route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003bc1b0} route53_test.go:93: Got zone list: [0xc0003bc1e0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003a0480 0xc00019c0e0} route53_test.go:187: Successfully added resource record set: {0xc000206100 0xc00019c100} --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003bc460} route53_test.go:93: Got zone list: [0xc0003bc490] route53_test.go:98: Got at least 1 zone in list:&{0xc0003a0480 0xc00019c188} route53_test.go:199: Successfully added resource record set: {0xc000206280 0xc00019c1a8} route53_test.go:118: Got 1 recordsets: [0xc0003bc630] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003bc760} route53_test.go:93: Got zone list: [0xc0003bc790] route53_test.go:98: Got at least 1 zone in list:&{0xc0003a0480 0xc00019c240} route53_test.go:221: Successfully added resource record set: {0xc000206400 0xc00019c260} route53_test.go:228: Correctly failed to add duplicate resource record {0xc000206400 0xc00019c260}: attempt to create duplicate rrset www11.example.com::A --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003bcaf0} route53_test.go:93: Got zone list: [0xc0003bcb20] route53_test.go:98: Got at least 1 zone in list:&{0xc0003a0480 0xc00019c318} route53_test.go:246: Successfully removed resource set {0xc000206600 0xc00019c338} after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003bcda0} route53_test.go:93: Got zone list: [0xc0003bcdd0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003a0480 0xc00019c3c0} route53_test.go:264: Successfully removed resource set {0xc000206780 0xc00019c3e0} after adding route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003bd080} route53_test.go:93: Got zone list: [0xc0003bd0b0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003a0480 0xc00019c470} commontests.go:192: Successfully added resource record set: {0xc000206900 0xc00019c490} commontests.go:45: Correctly replaced resource record {0xc000206900 0xc00019c490} -> {0xc000206a00 0xc00019c4e0} commontests.go:142: Got 1 recordsets: [0xc0003bd3c0] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003bd590} route53_test.go:93: Got zone list: [0xc0003bd5c0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003a0480 0xc00019c590} commontests.go:192: Successfully added resource record set: {0xc000206c80 0xc00019c5b0} commontests.go:70: Correctly replaced resource record {0xc000206c80 0xc00019c5b0} -> {0xc000206d80 0xc00019c600} commontests.go:142: Got 1 recordsets: [0xc0003bd8d0] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003bdaf0} route53_test.go:93: Got zone list: [0xc0003bdb20] route53_test.go:98: Got at least 1 zone in list:&{0xc0003a0480 0xc00019c6c0} commontests.go:192: Successfully added resource record set: {0xc000207000 0xc00019c6e0} commontests.go:142: Got 2 recordsets: [0xc0003bdd90 0xc0003bddb0] commontests.go:142: Got 2 recordsets: [0xc0003bde20 0xc0003bde40] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003bdfe0} route53_test.go:93: Got zone list: [0xc0003cc010] route53_test.go:98: Got at least 1 zone in list:&{0xc0003a0480 0xc00019c7e0} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53 0.067s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53/stubs [no test files] === RUN TestZonesList --- PASS: TestZonesList (0.00s) === RUN TestAdd --- PASS: TestAdd (0.00s) === RUN TestRemove --- PASS: TestRemove (0.00s) === RUN TestNewZone --- PASS: TestNewZone (0.00s) === RUN TestNewResourceRecordSet --- PASS: TestNewResourceRecordSet (0.00s) === RUN TestResourceRecordChangeset --- PASS: TestResourceRecordChangeset (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/do 0.022s === RUN TestZonesList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000288548 0xc0002912e0} clouddns_test.go:71: Got zone list: [0xc0005e1578] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000294d40 0xc0005e1548} --- PASS: TestZonesList (0.00s) === RUN TestZonesID clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000288548 0xc0002913e0} clouddns_test.go:71: Got zone list: [0xc0005e1620] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000294d40 0xc0005e15f0} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess clouddns_test.go:131: Getting zones clouddns_test.go:58: Got zones {0xc000288548 0xc0002914e0} clouddns_test.go:133: Got zones, making new Zone clouddns_test.go:149: Successfully added managed DNS zone: &{0xc000294e80 0xc0005e16c8} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000288548 0xc0002915a0} clouddns_test.go:71: Got zone list: [0xc0005e1770] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000294d40 0xc0005e1740} clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000288548 0xc0002916c0} clouddns_test.go:71: Got zone list: [0xc0005e1890] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000294d40 0xc0005e1860} clouddns_test.go:165: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000288548 0xc000291800} clouddns_test.go:71: Got zone list: [0xc0005e19e0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000294d40 0xc0005e19b0} clouddns_test.go:176: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:96: Got 1 recordsets: [<(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000288548 0xc000291960} clouddns_test.go:71: Got zone list: [0xc0005e1bd8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000294d40 0xc0005e1ba8} clouddns_test.go:198: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:205: Correctly failed to add duplicate resource record <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>: attempt to insert duplicate rrset {www11.example.com [10.10.10.10 169.20.20.20] 180 A} --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000288548 0xc000291ac0} clouddns_test.go:71: Got zone list: [0xc0005e1d58] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000294d40 0xc0005e1d28} clouddns_test.go:223: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000288548 0xc000291c00} clouddns_test.go:71: Got zone list: [0xc0005e1ea8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000294d40 0xc0005e1e78} clouddns_test.go:241: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000288548 0xc000291d60} clouddns_test.go:71: Got zone list: [0xc000302060] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000294d40 0xc000302030} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:45: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000288548 0xc000291ec0} clouddns_test.go:71: Got zone list: [0xc000302318] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000294d40 0xc0003022e8} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:70: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000288548 0xc000308040} clouddns_test.go:71: Got zone list: [0xc000302630] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000294d40 0xc000302600} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000288548 0xc000308220} clouddns_test.go:71: Got zone list: [0xc0003029f0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000294d40 0xc0003029c0} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns 0.078s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/interfaces [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/stubs [no test files] === RUN TestContract --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/openstack/designate 0.083s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/rrstype [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/tests [no test files] ? k8s.io/kops/examples/kops-api-example [no test files] === RUN TestBootstarapBuilder_Simple --- PASS: TestBootstarapBuilder_Simple (0.00s) PASS ok k8s.io/kops/nodeup/pkg/bootstrap 0.194s === RUN TestBuildAzure --- PASS: TestBuildAzure (0.00s) === RUN TestBuildAWSCustomNodeIPFamilies --- PASS: TestBuildAWSCustomNodeIPFamilies (0.00s) === RUN TestContainerdBuilder_Docker_19_03_13 I0512 07:42:14.922163 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:14.922281 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:14.922328 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:14.922395 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:14.922437 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:14.922463 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:14.922505 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:14.922560 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:14.922582 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:14.933466 8611 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_13 (0.01s) === RUN TestContainerdBuilder_Docker_19_03_14 I0512 07:42:14.936683 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:14.936723 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:14.936765 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:14.936802 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:14.936836 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:14.936858 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:14.936881 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:14.936907 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:14.936938 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:14.938274 8611 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_14 (0.01s) === RUN TestContainerdBuilder_Simple I0512 07:42:14.942830 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:14.942871 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:14.942908 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:14.942951 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:14.942989 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:14.943016 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:14.943044 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:14.943077 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:14.943101 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:14.944506 8611 zones.go:93] ListHostedZonesPages { } I0512 07:42:14.946411 8611 assetstore.go:108] Matching assets for "^bin/(containerd|ctr)": I0512 07:42:14.946431 8611 assetstore.go:111] bin/containerd I0512 07:42:14.946443 8611 assetstore.go:111] bin/containerd-shim I0512 07:42:14.946455 8611 assetstore.go:111] bin/containerd-shim-runc-v1 I0512 07:42:14.946465 8611 assetstore.go:111] bin/containerd-shim-runc-v2 I0512 07:42:14.946476 8611 assetstore.go:111] bin/containerd-stress I0512 07:42:14.946489 8611 assetstore.go:111] bin/ctr I0512 07:42:14.946552 8611 assetstore.go:108] Matching assets for "/runc\\.(amd64|arm64)$": I0512 07:42:14.946570 8611 assetstore.go:111] https://github.com/opencontainers/runc/releases/download/v1.1.0/runc.amd64 --- PASS: TestContainerdBuilder_Simple (0.01s) === RUN TestContainerdBuilder_Flatcar I0512 07:42:14.950424 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:14.950453 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:14.950474 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:14.950499 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:14.950520 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:14.950534 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:14.950548 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:14.950565 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:14.950577 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:14.951463 8611 zones.go:93] ListHostedZonesPages { } I0512 07:42:14.952452 8611 containerd.go:57] Detected Flatcar; won't install containerd --- PASS: TestContainerdBuilder_Flatcar (0.00s) === RUN TestContainerdBuilder_SkipInstall I0512 07:42:14.953358 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:14.953378 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:14.953395 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:14.953415 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:14.953472 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:14.953486 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:14.953499 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:14.953512 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:14.953523 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:14.954288 8611 zones.go:93] ListHostedZonesPages { } I0512 07:42:14.955472 8611 docker.go:60] SkipInstall is set to true; won't install Docker --- PASS: TestContainerdBuilder_SkipInstall (0.00s) === RUN TestContainerdBuilder_BuildFlags --- PASS: TestContainerdBuilder_BuildFlags (0.00s) === RUN TestContainerdConfig --- PASS: TestContainerdConfig (0.00s) === RUN TestAppendGPURuntimeContainerdConfig --- PASS: TestAppendGPURuntimeContainerdConfig (0.00s) === RUN TestDockerBuilder_19_03_11 I0512 07:42:14.956099 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:14.956120 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:14.956138 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:14.956157 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:14.956199 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:14.956216 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:14.956229 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:14.956243 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:14.956254 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:14.957088 8611 zones.go:93] ListHostedZonesPages { } I0512 07:42:14.958323 8611 assetstore.go:108] Matching assets for "^docker/": I0512 07:42:14.958337 8611 assetstore.go:111] docker/containerd I0512 07:42:14.958343 8611 assetstore.go:111] docker/containerd-shim I0512 07:42:14.958352 8611 assetstore.go:111] docker/ctr I0512 07:42:14.958360 8611 assetstore.go:111] docker/docker I0512 07:42:14.958365 8611 assetstore.go:111] docker/docker-init I0512 07:42:14.958370 8611 assetstore.go:111] docker/docker-proxy I0512 07:42:14.958374 8611 assetstore.go:111] docker/dockerd I0512 07:42:14.958379 8611 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_19_03_11 (0.00s) === RUN TestDockerBuilder_LogFlags I0512 07:42:14.961116 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:14.961169 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:14.961203 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:14.961233 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:14.961273 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:14.961287 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:14.961307 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:14.961321 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:14.961336 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:14.962184 8611 zones.go:93] ListHostedZonesPages { } I0512 07:42:14.963413 8611 assetstore.go:108] Matching assets for "^docker/": I0512 07:42:14.963427 8611 assetstore.go:111] docker/containerd I0512 07:42:14.963439 8611 assetstore.go:111] docker/containerd-shim I0512 07:42:14.963446 8611 assetstore.go:111] docker/ctr I0512 07:42:14.963451 8611 assetstore.go:111] docker/docker I0512 07:42:14.963456 8611 assetstore.go:111] docker/docker-init I0512 07:42:14.963461 8611 assetstore.go:111] docker/docker-proxy I0512 07:42:14.963466 8611 assetstore.go:111] docker/dockerd I0512 07:42:14.963475 8611 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_LogFlags (0.01s) === RUN TestDockerBuilder_SkipInstall I0512 07:42:14.968261 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:14.968297 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:14.968323 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:14.968350 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:14.968394 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:14.968408 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:14.968423 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:14.968441 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:14.968454 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:14.969319 8611 zones.go:93] ListHostedZonesPages { } I0512 07:42:14.970448 8611 docker.go:60] SkipInstall is set to true; won't install Docker --- PASS: TestDockerBuilder_SkipInstall (0.00s) === RUN TestDockerBuilder_HealthCheck I0512 07:42:14.970682 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:14.970705 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:14.970724 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:14.970745 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:14.970777 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:14.970791 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:14.970806 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:14.970819 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:14.970830 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:14.971608 8611 zones.go:93] ListHostedZonesPages { } I0512 07:42:14.972658 8611 assetstore.go:108] Matching assets for "^docker/": I0512 07:42:14.972673 8611 assetstore.go:111] docker/docker I0512 07:42:14.972679 8611 assetstore.go:111] docker/docker-containerd I0512 07:42:14.972685 8611 assetstore.go:111] docker/docker-containerd-ctr I0512 07:42:14.972690 8611 assetstore.go:111] docker/docker-containerd-shim I0512 07:42:14.972694 8611 assetstore.go:111] docker/docker-init I0512 07:42:14.972699 8611 assetstore.go:111] docker/docker-proxy I0512 07:42:14.972704 8611 assetstore.go:111] docker/docker-runc I0512 07:42:14.972709 8611 assetstore.go:111] docker/dockerd --- PASS: TestDockerBuilder_HealthCheck (0.00s) === RUN TestDockerBuilder_BuildFlags --- PASS: TestDockerBuilder_BuildFlags (0.00s) === RUN TestContainerdHooksBuilder I0512 07:42:14.976014 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:14.976056 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:14.976078 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:14.976103 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:14.976124 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:14.976140 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:14.976156 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:14.976173 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:14.976185 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:14.978574 8611 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdHooksBuilder (0.01s) === RUN TestDockerHooksBuilder I0512 07:42:14.981565 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:14.981622 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:14.981664 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:14.981713 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:14.981746 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:14.981767 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:14.981790 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:14.981816 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:14.981838 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:14.983869 8611 zones.go:93] ListHostedZonesPages { } --- PASS: TestDockerHooksBuilder (0.00s) === RUN TestKopsControllerBuilder I0512 07:42:14.985936 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:14.985982 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:14.986003 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:14.986023 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:14.986042 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:14.986055 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:14.986070 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:14.986084 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:14.986095 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:14.987232 8611 zones.go:93] ListHostedZonesPages { } --- PASS: TestKopsControllerBuilder (0.00s) === RUN Test_KubeAPIServer_BuildFlags --- PASS: Test_KubeAPIServer_BuildFlags (0.00s) === RUN TestKubeAPIServerBuilder I0512 07:42:14.991807 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:14.991841 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:14.991860 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:14.991880 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:14.991904 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:14.991919 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:14.991933 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:14.991951 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:14.991963 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:14.993159 8611 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilder (0.01s) === RUN TestDeddicatedAPIServerBuilder I0512 07:42:15.005791 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:15.005862 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:15.005904 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:15.005948 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:15.005988 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:15.006018 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:15.006047 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:15.006077 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:15.006099 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:15.008790 8611 zones.go:93] ListHostedZonesPages { } --- PASS: TestDeddicatedAPIServerBuilder (0.01s) === RUN TestWithoutEtcdEventsAPIServerBuilder I0512 07:42:15.018750 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:15.018813 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:15.018851 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:15.018894 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:15.018932 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:15.018963 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:15.018993 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:15.019022 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:15.019045 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:15.021488 8611 zones.go:93] ListHostedZonesPages { } --- PASS: TestWithoutEtcdEventsAPIServerBuilder (0.01s) === RUN TestAwsIamAuthenticator I0512 07:42:15.030970 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:15.031027 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:15.031064 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:15.031103 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:15.031156 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:15.031187 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:15.031223 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:15.031250 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:15.031271 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:15.033779 8611 zones.go:93] ListHostedZonesPages { } --- PASS: TestAwsIamAuthenticator (0.01s) === RUN TestKubeAPIServerBuilderAMD64 I0512 07:42:15.043988 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:15.044048 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:15.044080 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:15.044114 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:15.044139 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:15.044157 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:15.044175 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:15.044195 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:15.044211 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:15.046326 8611 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderAMD64 (1.51s) === RUN TestKubeAPIServerBuilderARM64 I0512 07:42:16.557212 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:16.557269 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:16.557297 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:16.557324 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:16.557347 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:16.557360 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:16.557375 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:16.557396 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:16.557418 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:16.558841 8611 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderARM64 (0.08s) === RUN TestKubeControllerManagerBuilder I0512 07:42:16.638188 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:16.638305 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:16.638371 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:16.638421 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:16.638460 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:16.638489 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:16.638520 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:16.638552 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:16.638578 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:16.641467 8611 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilder (0.01s) === RUN TestKubeControllerManagerBuilderAMD64 I0512 07:42:16.652026 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:16.652105 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:16.652150 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:16.652206 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:16.652252 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:16.652280 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:16.652319 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:16.652352 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:16.652381 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:16.655235 8611 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderAMD64 (0.08s) === RUN TestKubeControllerManagerBuilderARM64 I0512 07:42:16.728436 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:16.728525 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:16.728572 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:16.728626 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:16.728672 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:16.728701 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:16.728732 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:16.728763 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:16.728788 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:16.731515 8611 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderARM64 (0.07s) === RUN TestKubeProxyBuilder_buildPod === RUN TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 --- PASS: TestKubeProxyBuilder_buildPod (0.00s) --- PASS: TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 (0.00s) === RUN TestKubeProxyBuilder I0512 07:42:16.801737 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:16.801792 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:16.801818 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:16.801845 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:16.801881 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:16.801894 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:16.801908 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:16.801936 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:16.801949 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:16.803281 8611 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilder (0.00s) === RUN TestKubeProxyBuilderAMD64 I0512 07:42:16.806395 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:16.806450 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:16.806471 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:16.806494 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:16.806516 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:16.806529 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:16.806543 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:16.806561 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:16.806573 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:16.808257 8611 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderAMD64 (0.07s) === RUN TestKubeProxyBuilderARM64 I0512 07:42:16.872927 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:16.872986 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:16.873012 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:16.873039 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:16.873060 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:16.873073 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:16.873086 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:16.873104 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:16.873119 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:16.874378 8611 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderARM64 (0.07s) === RUN TestParseDefault --- PASS: TestParseDefault (0.00s) === RUN TestParse --- PASS: TestParse (0.00s) === RUN TestKubeSchedulerBuilder I0512 07:42:16.943592 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:16.943681 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:16.943740 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:16.943787 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:16.943845 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:16.943896 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:16.943917 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:16.943963 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:16.943988 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:16.946885 8611 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeSchedulerBuilder (0.01s) === RUN TestKubeSchedulerBuilderAMD64 I0512 07:42:16.954390 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:16.954456 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:16.954517 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:16.954590 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:16.954656 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:16.954709 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:16.954733 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:16.954776 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:16.954798 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:16.957395 8611 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeSchedulerBuilderAMD64 (0.07s) === RUN TestKubeSchedulerBuilderARM64 I0512 07:42:17.025996 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:17.026069 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:17.026114 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:17.026158 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:17.026195 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:17.026223 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:17.026251 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:17.026279 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:17.026305 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:17.028967 8611 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeSchedulerBuilderARM64 (0.07s) === RUN TestKubectlBuilder I0512 07:42:17.099755 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:17.099841 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:17.099885 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:17.099934 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:17.099977 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:17.100009 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:17.100040 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:17.100072 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:17.100098 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:17.102830 8611 zones.go:93] ListHostedZonesPages { } I0512 07:42:17.108400 8611 assetstore.go:159] Resolved asset kubectl: to /path/to/kubectl/asset --- PASS: TestKubectlBuilder (0.01s) === RUN Test_InstanceGroupKubeletMerge --- PASS: Test_InstanceGroupKubeletMerge (0.00s) === RUN TestTaintsApplied --- PASS: TestTaintsApplied (0.00s) === RUN Test_RunKubeletBuilder I0512 07:42:17.111226 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:17.111293 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:17.111338 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:17.111384 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:17.111418 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:17.111449 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:17.111479 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:17.111508 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:17.111532 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:17.113710 8611 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilder (0.01s) === RUN Test_RunKubeletBuilderWarmPool I0512 07:42:17.119706 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:17.119775 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:17.119814 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:17.119857 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:17.119895 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:17.119927 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:17.119956 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:17.119984 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:17.120006 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:17.122114 8611 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilderWarmPool (0.01s) === RUN Test_BuildComponentConfigFile --- PASS: Test_BuildComponentConfigFile (0.00s) === RUN TestProtokubeBuilder I0512 07:42:17.127995 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:17.128069 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:17.128110 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:17.128163 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:17.128200 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:17.128229 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:17.128258 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:17.128286 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:17.128310 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:17.130869 8611 zones.go:93] ListHostedZonesPages { } I0512 07:42:17.133404 8611 assetstore.go:108] Matching assets for "protokube$": I0512 07:42:17.133433 8611 assetstore.go:111] /opt/kops/bin/protokube I0512 07:42:17.133449 8611 assetstore.go:129] Found single matching asset for expr "protokube$": "protokube" I0512 07:42:17.133482 8611 assetstore.go:108] Matching assets for "channels$": I0512 07:42:17.133497 8611 assetstore.go:111] /opt/kops/bin/channels I0512 07:42:17.133509 8611 assetstore.go:129] Found single matching asset for expr "channels$": "channels" --- PASS: TestProtokubeBuilder (0.01s) === RUN TestSecretBuilder I0512 07:42:17.135828 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:17.135890 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:17.135925 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:17.135963 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:17.135994 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:17.136021 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:17.136062 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:17.136087 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:17.136106 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:17.138359 8611 zones.go:93] ListHostedZonesPages { } --- PASS: TestSecretBuilder (0.01s) === RUN TestUpdateServiceBuilderAutomaticUpgrade I0512 07:42:17.141491 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:17.141554 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:17.141585 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:17.141618 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:17.141647 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:17.141671 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:17.141693 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:17.141715 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:17.141735 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:17.143778 8611 zones.go:93] ListHostedZonesPages { } I0512 07:42:17.145882 8611 update_service.go:96] Detected OS {deb ubuntu focal 20.04}; installing unattended-upgrades package --- PASS: TestUpdateServiceBuilderAutomaticUpgrade (0.00s) === RUN TestUpdateServiceBuilderExternal I0512 07:42:17.146489 8611 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:17.146546 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:17.146577 8611 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:17.146610 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:17.146638 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:17.146663 8611 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:17.146686 8611 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:17.146708 8611 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:17.146725 8611 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:17.148635 8611 zones.go:93] ListHostedZonesPages { } I0512 07:42:17.150519 8611 update_service.go:91] UpdatePolicy requests automatic updates; skipping installation of package "unattended-upgrades" --- PASS: TestUpdateServiceBuilderExternal (0.00s) PASS ok k8s.io/kops/nodeup/pkg/model 2.303s ? k8s.io/kops/nodeup/pkg/model/dns [no test files] ? k8s.io/kops/nodeup/pkg/model/networking [no test files] ? k8s.io/kops/nodeup/pkg/model/resources [no test files] ? k8s.io/kops/pkg/acls [no test files] ? k8s.io/kops/pkg/acls/gce [no test files] === RUN Test_Strategy --- PASS: Test_Strategy (0.00s) === RUN Test_In_StateStore --- PASS: Test_In_StateStore (0.00s) PASS ok k8s.io/kops/pkg/acls/s3 0.047s === RUN TestWarmPoolSpec_IsEnabled === RUN TestWarmPoolSpec_IsEnabled/nil === RUN TestWarmPoolSpec_IsEnabled/empty === RUN TestWarmPoolSpec_IsEnabled/1 === RUN TestWarmPoolSpec_IsEnabled/0 --- PASS: TestWarmPoolSpec_IsEnabled (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/nil (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/empty (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/1 (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/0 (0.00s) === RUN TestWarmPoolSpec_ResolveDefaults === RUN TestWarmPoolSpec_ResolveDefaults/MinSize === RUN TestWarmPoolSpec_ResolveDefaults/MaxSize === RUN TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook --- PASS: TestWarmPoolSpec_ResolveDefaults (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MinSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MaxSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook (0.00s) === RUN Test_ParseInstanceGroupRole --- PASS: Test_ParseInstanceGroupRole (0.00s) === RUN TestParseConfigYAML === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{}" --- PASS: TestParseConfigYAML (0.01s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" (0.01s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{}" (0.00s) === RUN TestWeaveParseConfigYAML === RUN TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" === RUN TestWeaveParseConfigYAML/"networking:_{}" --- PASS: TestWeaveParseConfigYAML (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{}" (0.00s) === RUN Test_SemverOrdering --- PASS: Test_SemverOrdering (0.00s) PASS ok k8s.io/kops/pkg/apis/kops 0.136s === RUN TestRoundTripTypes --- PASS: TestRoundTripTypes (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/install 0.107s === RUN TestUseCiliumEtcd --- PASS: TestUseCiliumEtcd (0.00s) === RUN Test_FindSubnet --- PASS: Test_FindSubnet (0.00s) === RUN Test_FindZonesForInstanceGroup --- PASS: Test_FindZonesForInstanceGroup (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/model 0.098s ? k8s.io/kops/pkg/apis/kops/registry [no test files] === RUN TestGetNodeRole === RUN TestGetNodeRole/RoleNone === RUN TestGetNodeRole/RoleNewerLabel === RUN TestGetNodeRole/RoleOlderLabel --- PASS: TestGetNodeRole (0.00s) --- PASS: TestGetNodeRole/RoleNone (0.00s) --- PASS: TestGetNodeRole/RoleNewerLabel (0.00s) --- PASS: TestGetNodeRole/RoleOlderLabel (0.00s) === RUN Test_ParseKubernetesVersion === RUN Test_ParseKubernetesVersion/1.3.7 === RUN Test_ParseKubernetesVersion/v1.4.0-beta.8 === RUN Test_ParseKubernetesVersion/1.5.0 === RUN Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ === RUN Test_ParseKubernetesVersion/#00 E0512 07:41:56.874638 4192 versions.go:96] unable to parse Kubernetes version "" === RUN Test_ParseKubernetesVersion/abc E0512 07:41:56.874774 4192 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: Test_ParseKubernetesVersion (0.00s) --- PASS: Test_ParseKubernetesVersion/1.3.7 (0.00s) --- PASS: Test_ParseKubernetesVersion/v1.4.0-beta.8 (0.00s) --- PASS: Test_ParseKubernetesVersion/1.5.0 (0.00s) --- PASS: Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ (0.00s) --- PASS: Test_ParseKubernetesVersion/#00 (0.00s) --- PASS: Test_ParseKubernetesVersion/abc (0.00s) === RUN Test_IsKubernetesGTEWithPatch --- PASS: Test_IsKubernetesGTEWithPatch (0.00s) === RUN Test_IsKubernetesGTEWithoutPatch --- PASS: Test_IsKubernetesGTEWithoutPatch (0.00s) === RUN Test_IsKubernetesGTEWithPre --- PASS: Test_IsKubernetesGTEWithPre (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/util 0.056s ? k8s.io/kops/pkg/apis/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha3 [no test files] === RUN TestAWSValidateExternalCloudConfig --- PASS: TestAWSValidateExternalCloudConfig (0.00s) === RUN TestValidateInstanceGroupSpec I0512 07:42:09.422877 7691 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0512 07:42:09.423028 7691 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0512 07:42:09.423092 7691 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0512 07:42:09.423150 7691 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0512 07:42:09.423202 7691 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestValidateInstanceGroupSpec (0.00s) === RUN TestMixedInstancePolicies I0512 07:42:09.423479 7691 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0512 07:42:09.423543 7691 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0512 07:42:09.423596 7691 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0512 07:42:09.423652 7691 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0512 07:42:09.423707 7691 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0512 07:42:09.423775 7691 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0512 07:42:09.423834 7691 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0512 07:42:09.423883 7691 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0512 07:42:09.423941 7691 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0512 07:42:09.423993 7691 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0512 07:42:09.424044 7691 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0512 07:42:09.424126 7691 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0512 07:42:09.424178 7691 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0512 07:42:09.424233 7691 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0512 07:42:09.424291 7691 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0512 07:42:09.424372 7691 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0512 07:42:09.424425 7691 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0512 07:42:09.424471 7691 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0512 07:42:09.424550 7691 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestMixedInstancePolicies (0.00s) === RUN TestInstanceMetadataOptions I0512 07:42:09.424828 7691 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } I0512 07:42:09.424912 7691 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } --- PASS: TestInstanceMetadataOptions (0.00s) === RUN TestLoadBalancerSubnets --- PASS: TestLoadBalancerSubnets (0.00s) === RUN TestAWSAuthentication --- PASS: TestAWSAuthentication (0.00s) === RUN TestValidEtcdChanges --- PASS: TestValidEtcdChanges (0.00s) === RUN TestValidateInstanceProfile --- PASS: TestValidateInstanceProfile (0.00s) === RUN TestValidMasterInstanceGroup --- PASS: TestValidMasterInstanceGroup (0.00s) === RUN TestValidBootDevice --- PASS: TestValidBootDevice (0.00s) === RUN TestValidNodeLabels --- PASS: TestValidNodeLabels (0.00s) === RUN TestValidateIGCloudLabels --- PASS: TestValidateIGCloudLabels (0.00s) === RUN TestIGCloudLabelIsIGName --- PASS: TestIGCloudLabelIsIGName (0.00s) === RUN TestValidTaints --- PASS: TestValidTaints (0.00s) === RUN TestIGUpdatePolicy === RUN TestIGUpdatePolicy/missing === RUN TestIGUpdatePolicy/automatic === RUN TestIGUpdatePolicy/external === RUN TestIGUpdatePolicy/empty === RUN TestIGUpdatePolicy/unknown --- PASS: TestIGUpdatePolicy (0.00s) --- PASS: TestIGUpdatePolicy/missing (0.00s) --- PASS: TestIGUpdatePolicy/automatic (0.00s) --- PASS: TestIGUpdatePolicy/external (0.00s) --- PASS: TestIGUpdatePolicy/empty (0.00s) --- PASS: TestIGUpdatePolicy/unknown (0.00s) === RUN TestValidInstanceGroup --- PASS: TestValidInstanceGroup (0.00s) === RUN Test_ValidateTopology --- PASS: Test_ValidateTopology (0.00s) === RUN Test_Validate_DNS --- PASS: Test_Validate_DNS (0.00s) === RUN TestValidateCIDR --- PASS: TestValidateCIDR (0.00s) === RUN TestValidateSubnets --- PASS: TestValidateSubnets (0.00s) === RUN TestValidateKubeAPIServer --- PASS: TestValidateKubeAPIServer (0.00s) === RUN Test_Validate_DockerConfig_Storage --- PASS: Test_Validate_DockerConfig_Storage (0.00s) === RUN Test_Validate_Networking_Flannel --- PASS: Test_Validate_Networking_Flannel (0.00s) === RUN Test_Validate_AdditionalPolicies --- PASS: Test_Validate_AdditionalPolicies (0.00s) === RUN Test_Validate_Calico === RUN Test_Validate_Calico/empty_specs === RUN Test_Validate_Calico/positive_Typha_replica_count === RUN Test_Validate_Calico/negative_Typha_replica_count === RUN Test_Validate_Calico/with_etcd_version === RUN Test_Validate_Calico/IPv4_autodetection_method === RUN Test_Validate_Calico/IPv6_autodetection_method === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name === RUN Test_Validate_Calico/AWS_source/destination_checks_off === RUN Test_Validate_Calico/AWS_source/destination_checks_enabled === RUN Test_Validate_Calico/AWS_source/destination_checks_disabled === RUN Test_Validate_Calico/AWS_source/destination_checks_left_as_is === RUN Test_Validate_Calico/encapsulation_none_with_IPv4 === RUN Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 === RUN Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 === RUN Test_Validate_Calico/unknown_Calico_IPIP_mode === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPv6_without_encapsulation --- PASS: Test_Validate_Calico (0.00s) --- PASS: Test_Validate_Calico/empty_specs (0.00s) --- PASS: Test_Validate_Calico/positive_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/negative_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/with_etcd_version (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_off (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_enabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_disabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_left_as_is (0.00s) --- PASS: Test_Validate_Calico/encapsulation_none_with_IPv4 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/unknown_Calico_IPIP_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPv6_without_encapsulation (0.00s) === RUN Test_Validate_Cilium --- PASS: Test_Validate_Cilium (0.00s) === RUN Test_Validate_RollingUpdate --- PASS: Test_Validate_RollingUpdate (0.00s) === RUN Test_Validate_NodeLocalDNS --- PASS: Test_Validate_NodeLocalDNS (0.00s) === RUN Test_Validate_CloudConfiguration === RUN Test_Validate_CloudConfiguration/neither === RUN Test_Validate_CloudConfiguration/all_false === RUN Test_Validate_CloudConfiguration/all_true === RUN Test_Validate_CloudConfiguration/os_false === RUN Test_Validate_CloudConfiguration/os_true === RUN Test_Validate_CloudConfiguration/all_false,_os_false === RUN Test_Validate_CloudConfiguration/all_false,_os_true === RUN Test_Validate_CloudConfiguration/all_true,_os_false === RUN Test_Validate_CloudConfiguration/all_true,_os_true --- PASS: Test_Validate_CloudConfiguration (0.00s) --- PASS: Test_Validate_CloudConfiguration/neither (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_true (0.00s) === RUN TestValidateSAExternalPermissions === RUN TestValidateSAExternalPermissions/Duplicate_SA === RUN TestValidateSAExternalPermissions/Missing_permissions === RUN TestValidateSAExternalPermissions/Setting_both_arn_and_inline === RUN TestValidateSAExternalPermissions/Empty_SA_name === RUN TestValidateSAExternalPermissions/Empty_SA_namespace --- PASS: TestValidateSAExternalPermissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Duplicate_SA (0.00s) --- PASS: TestValidateSAExternalPermissions/Missing_permissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Setting_both_arn_and_inline (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_name (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_namespace (0.00s) === RUN Test_Validate_Nvdia --- PASS: Test_Validate_Nvdia (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/validation 0.143s ? k8s.io/kops/pkg/apis/nodeup [no test files] === RUN TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags (0.00s) === RUN TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges --- PASS: TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges (0.00s) === RUN TestRemapEmptySection --- PASS: TestRemapEmptySection (0.00s) === RUN Test_BuildVFSPath E0512 07:41:59.107187 4968 copyfile.go:205] Unable to determine VFS path from supplied URL: https://foo/k8s-for-greeks-kops/kubernetes-release/release/v1.7.2/bin/linux/amd64/kubectl E0512 07:41:59.107237 4968 copyfile.go:206] S3, Google Cloud Storage, and File Paths are supported. E0512 07:41:59.107247 4968 copyfile.go:207] For S3, please make sure that the supplied file repository URL adhere to S3 naming conventions, https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region. E0512 07:41:59.107258 4968 copyfile.go:208] For GCS, please make sure that the supplied file repository URL adheres to https://storage.googleapis.com/ --- PASS: Test_BuildVFSPath (0.00s) PASS ok k8s.io/kops/pkg/assets 0.048s === RUN TestBackoff --- PASS: TestBackoff (0.00s) PASS ok k8s.io/kops/pkg/backoff 0.023s ? k8s.io/kops/pkg/bootstrap [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/simple [no test files] ? k8s.io/kops/pkg/client/simple/api [no test files] === RUN TestSSHCredentialStoreOnConfigBase --- PASS: TestSSHCredentialStoreOnConfigBase (0.00s) === RUN TestSSHCredentialStoreOnOwnCFS --- PASS: TestSSHCredentialStoreOnOwnCFS (0.00s) PASS ok k8s.io/kops/pkg/client/simple/vfsclientset 0.175s === RUN TestToAzureVMName === RUN TestToAzureVMName/test_case_0 === RUN TestToAzureVMName/test_case_1 --- PASS: TestToAzureVMName (0.00s) --- PASS: TestToAzureVMName/test_case_0 (0.00s) --- PASS: TestToAzureVMName/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/cloudinstances 0.063s ? k8s.io/kops/pkg/clouds [no test files] ? k8s.io/kops/pkg/clusteraddons [no test files] === RUN TestSetClusterBadInput --- PASS: TestSetClusterBadInput (0.00s) === RUN TestSetClusterFields --- PASS: TestSetClusterFields (0.01s) === RUN TestSetCiliumFields --- PASS: TestSetCiliumFields (0.00s) === RUN TestSetInstanceGroupsBadInput --- PASS: TestSetInstanceGroupsBadInput (0.00s) === RUN TestSetInstanceGroupsFields --- PASS: TestSetInstanceGroupsFields (0.00s) === RUN TestUnsetClusterBadInput --- PASS: TestUnsetClusterBadInput (0.00s) === RUN TestUnsetClusterFields --- PASS: TestUnsetClusterFields (0.00s) === RUN TestUnsetCiliumFields --- PASS: TestUnsetCiliumFields (0.00s) === RUN TestUnsetInstanceGroupsBadInput --- PASS: TestUnsetInstanceGroupsBadInput (0.00s) === RUN TestUnsetInstanceGroupsFields --- PASS: TestUnsetInstanceGroupsFields (0.00s) PASS ok k8s.io/kops/pkg/commands 0.265s ? k8s.io/kops/pkg/commands/commandutils [no test files] ? k8s.io/kops/pkg/commands/helpers [no test files] === RUN TestGetStructVal --- PASS: TestGetStructVal (0.00s) === RUN TestWrongStructField --- PASS: TestWrongStructField (0.00s) PASS ok k8s.io/kops/pkg/configbuilder 0.050s ? k8s.io/kops/pkg/configserver [no test files] === RUN Test_Diff_1 --- PASS: Test_Diff_1 (0.00s) === RUN Test_Diff_2 --- PASS: Test_Diff_2 (0.00s) === RUN Test_Diff_3 --- PASS: Test_Diff_3 (0.00s) === RUN Test_Diff_ChangedLine --- PASS: Test_Diff_ChangedLine (0.00s) === RUN Test_Diff_4 --- PASS: Test_Diff_4 (0.00s) PASS ok k8s.io/kops/pkg/diff 0.048s === RUN TestIsGossipHostname --- PASS: TestIsGossipHostname (0.00s) PASS ok k8s.io/kops/pkg/dns 0.042s ? k8s.io/kops/pkg/dump [no test files] === RUN TestHasExtraFields --- PASS: TestHasExtraFields (0.01s) PASS ok k8s.io/kops/pkg/edit 0.134s === RUN TestFlagToFalse I0512 07:41:57.272303 4401 featureflag_test.go:33] Created flag Unittest1 I0512 07:41:57.272402 4401 featureflag.go:162] FeatureFlag "UnitTest1"=false I0512 07:41:57.272418 4401 featureflag.go:162] FeatureFlag "UnitTest1"=true --- PASS: TestFlagToFalse (0.00s) === RUN TestSetenv I0512 07:41:57.272508 4401 featureflag.go:162] FeatureFlag "UnitTest2"=false --- PASS: TestSetenv (0.00s) PASS ok k8s.io/kops/pkg/featureflag 0.050s === RUN TestBuildKCMFlags --- PASS: TestBuildKCMFlags (0.00s) === RUN TestKubeletConfigSpec --- PASS: TestKubeletConfigSpec (0.00s) === RUN TestBuildAPIServerFlags --- PASS: TestBuildAPIServerFlags (0.00s) PASS ok k8s.io/kops/pkg/flagbuilder 0.127s === RUN TestRenderInstanceGroupZones W0512 07:42:00.200019 5235 instancegroup.go:42] error fetching zones for instancegroup: cannot find subnet "badsubnet" (declared in instance group "", not found in cluster) --- PASS: TestRenderInstanceGroupZones (0.00s) === RUN TestRenderInstanceGroupSubnets --- PASS: TestRenderInstanceGroupSubnets (0.00s) PASS ok k8s.io/kops/pkg/formatter 0.131s === RUN TestRollingUpdateDisabledSurgeOS rollingupdate_os_test.go:100: Failing in new release PR when build is not yet published --- SKIP: TestRollingUpdateDisabledSurgeOS (0.00s) === RUN TestRollingUpdateAllNeedUpdate I0512 07:42:27.911277 8889 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0512 07:42:27.911332 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:27.912500 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:27.912517 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:27.917684 8889 instancegroups.go:501] Cluster validated. I0512 07:42:27.917718 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:27.917727 8889 instancegroups.go:501] Cluster validated. I0512 07:42:27.917732 8889 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0512 07:42:27.918868 8889 instancegroups.go:398] Draining the node: "master-1a.local". I0512 07:42:27.919231 8889 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0512 07:42:27.919246 8889 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0512 07:42:27.919257 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:27.920440 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:27.920479 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:27.925698 8889 instancegroups.go:501] Cluster validated. I0512 07:42:27.925820 8889 instancegroups.go:398] Draining the node: "master-1b.local". I0512 07:42:27.926869 8889 instancegroups.go:417] deleting node "master-1b.local" from kubernetes I0512 07:42:27.926915 8889 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0512 07:42:27.926943 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:27.928127 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:27.928148 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:27.933419 8889 instancegroups.go:501] Cluster validated. I0512 07:42:27.933470 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:27.933482 8889 instancegroups.go:501] Cluster validated. I0512 07:42:27.933490 8889 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0512 07:42:27.934564 8889 instancegroups.go:398] Draining the node: "node-1a.local". I0512 07:42:27.935445 8889 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0512 07:42:27.935492 8889 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0512 07:42:27.935516 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:27.936727 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:27.936747 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:27.941993 8889 instancegroups.go:501] Cluster validated. I0512 07:42:27.942050 8889 instancegroups.go:398] Draining the node: "node-1b.local". I0512 07:42:27.946190 8889 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0512 07:42:27.946227 8889 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0512 07:42:27.946253 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:27.947489 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:27.947514 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:27.952682 8889 instancegroups.go:501] Cluster validated. I0512 07:42:27.952721 8889 instancegroups.go:398] Draining the node: "node-1c.local". I0512 07:42:27.953263 8889 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0512 07:42:27.953283 8889 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0512 07:42:27.953296 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:27.954544 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:27.954566 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:27.959920 8889 instancegroups.go:501] Cluster validated. I0512 07:42:27.959943 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:27.959956 8889 instancegroups.go:501] Cluster validated. I0512 07:42:27.959977 8889 instancegroups.go:309] Tainting 3 nodes in "node-2" instancegroup. I0512 07:42:27.960652 8889 instancegroups.go:398] Draining the node: "node-2a.local". I0512 07:42:27.961184 8889 instancegroups.go:417] deleting node "node-2a.local" from kubernetes I0512 07:42:27.961206 8889 instancegroups.go:589] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0512 07:42:27.961222 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:27.962448 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:27.962494 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:27.967792 8889 instancegroups.go:501] Cluster validated. I0512 07:42:27.967845 8889 instancegroups.go:398] Draining the node: "node-2b.local". I0512 07:42:27.968558 8889 instancegroups.go:417] deleting node "node-2b.local" from kubernetes I0512 07:42:27.968584 8889 instancegroups.go:589] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0512 07:42:27.968602 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:27.969809 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:27.969856 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:27.975133 8889 instancegroups.go:501] Cluster validated. I0512 07:42:27.975179 8889 instancegroups.go:398] Draining the node: "node-2c.local". I0512 07:42:27.975931 8889 instancegroups.go:417] deleting node "node-2c.local" from kubernetes I0512 07:42:27.975960 8889 instancegroups.go:589] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0512 07:42:27.975982 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:27.977197 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:27.977249 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:27.982480 8889 instancegroups.go:501] Cluster validated. I0512 07:42:27.982514 8889 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdate (0.07s) === RUN TestRollingUpdateAllNeedUpdateCloudonly I0512 07:42:27.983115 8889 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0512 07:42:27.983146 8889 instancegroups.go:435] waiting for 1ms after terminating instance W0512 07:42:27.984357 8889 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0512 07:42:27.984414 8889 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0512 07:42:27.984457 8889 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0512 07:42:27.984497 8889 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0512 07:42:27.984532 8889 instancegroups.go:435] waiting for 1ms after terminating instance W0512 07:42:27.985733 8889 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0512 07:42:27.985784 8889 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0512 07:42:27.985800 8889 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0512 07:42:27.985825 8889 instancegroups.go:435] waiting for 1ms after terminating instance W0512 07:42:27.987033 8889 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0512 07:42:27.987068 8889 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0512 07:42:27.987114 8889 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0512 07:42:27.987128 8889 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0512 07:42:27.987160 8889 instancegroups.go:435] waiting for 1ms after terminating instance W0512 07:42:27.988342 8889 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0512 07:42:27.988394 8889 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0512 07:42:27.988412 8889 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0512 07:42:27.988439 8889 instancegroups.go:435] waiting for 1ms after terminating instance W0512 07:42:27.989647 8889 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0512 07:42:27.989698 8889 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0512 07:42:27.989713 8889 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0512 07:42:27.989737 8889 instancegroups.go:435] waiting for 1ms after terminating instance W0512 07:42:27.990954 8889 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0512 07:42:27.990982 8889 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0512 07:42:27.991028 8889 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0512 07:42:27.991042 8889 instancegroups.go:589] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0512 07:42:27.991070 8889 instancegroups.go:435] waiting for 1ms after terminating instance W0512 07:42:27.992215 8889 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0512 07:42:27.992268 8889 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0512 07:42:27.992284 8889 instancegroups.go:589] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0512 07:42:27.992309 8889 instancegroups.go:435] waiting for 1ms after terminating instance W0512 07:42:27.993505 8889 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0512 07:42:27.993555 8889 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0512 07:42:27.993573 8889 instancegroups.go:589] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0512 07:42:27.993598 8889 instancegroups.go:435] waiting for 1ms after terminating instance W0512 07:42:27.994748 8889 instancegroups.go:466] Not validating cluster as cloudonly flag is set. I0512 07:42:27.994779 8889 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateCloudonly (0.01s) === RUN TestRollingUpdateAllNeedUpdateNoFailOnValidate I0512 07:42:27.995068 8889 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0512 07:42:27.995099 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:27.996271 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:27.996312 8889 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0512 07:42:27.996334 8889 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0512 07:42:27.996388 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:27.996414 8889 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0512 07:42:27.996430 8889 instancegroups.go:477] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0512 07:42:27.996446 8889 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0512 07:42:27.997217 8889 instancegroups.go:398] Draining the node: "master-1a.local". I0512 07:42:27.998025 8889 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0512 07:42:27.998056 8889 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0512 07:42:27.998081 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:27.999207 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:27.999240 8889 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0512 07:42:27.999257 8889 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0512 07:42:27.999290 8889 instancegroups.go:398] Draining the node: "master-1b.local". I0512 07:42:28.000208 8889 instancegroups.go:417] deleting node "master-1b.local" from kubernetes I0512 07:42:28.000239 8889 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0512 07:42:28.000261 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.001418 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.001452 8889 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0512 07:42:28.001470 8889 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0512 07:42:28.001489 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.001506 8889 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0512 07:42:28.001529 8889 instancegroups.go:477] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0512 07:42:28.001546 8889 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0512 07:42:28.002436 8889 instancegroups.go:398] Draining the node: "node-1a.local". I0512 07:42:28.003258 8889 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0512 07:42:28.003293 8889 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0512 07:42:28.003316 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.004435 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.004465 8889 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0512 07:42:28.004502 8889 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0512 07:42:28.004538 8889 instancegroups.go:398] Draining the node: "node-1b.local". I0512 07:42:28.005335 8889 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0512 07:42:28.005366 8889 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0512 07:42:28.005389 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.006535 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.006567 8889 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0512 07:42:28.006584 8889 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0512 07:42:28.006624 8889 instancegroups.go:398] Draining the node: "node-1c.local". I0512 07:42:28.007430 8889 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0512 07:42:28.007461 8889 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0512 07:42:28.007486 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.008602 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.008631 8889 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0512 07:42:28.008647 8889 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0512 07:42:28.008661 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.008678 8889 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0512 07:42:28.008690 8889 instancegroups.go:477] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0512 07:42:28.008704 8889 instancegroups.go:309] Tainting 3 nodes in "node-2" instancegroup. I0512 07:42:28.009618 8889 instancegroups.go:398] Draining the node: "node-2a.local". I0512 07:42:28.010338 8889 instancegroups.go:417] deleting node "node-2a.local" from kubernetes I0512 07:42:28.010370 8889 instancegroups.go:589] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0512 07:42:28.010392 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.011538 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.011571 8889 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0512 07:42:28.011589 8889 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0512 07:42:28.011624 8889 instancegroups.go:398] Draining the node: "node-2b.local". I0512 07:42:28.012379 8889 instancegroups.go:417] deleting node "node-2b.local" from kubernetes I0512 07:42:28.012411 8889 instancegroups.go:589] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0512 07:42:28.012434 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.013550 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.013614 8889 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0512 07:42:28.013633 8889 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0512 07:42:28.013669 8889 instancegroups.go:398] Draining the node: "node-2c.local". I0512 07:42:28.014414 8889 instancegroups.go:417] deleting node "node-2c.local" from kubernetes I0512 07:42:28.014448 8889 instancegroups.go:589] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0512 07:42:28.014500 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.015657 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.015691 8889 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0512 07:42:28.015712 8889 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0512 07:42:28.015728 8889 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateNoFailOnValidate (0.02s) === RUN TestRollingUpdateNoneNeedUpdate I0512 07:42:28.016006 8889 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdate (0.00s) === RUN TestRollingUpdateNoneNeedUpdateWithForce I0512 07:42:28.016249 8889 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0512 07:42:28.016278 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.017391 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.017416 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.022622 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.022689 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.022715 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.022727 8889 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0512 07:42:28.023433 8889 instancegroups.go:398] Draining the node: "master-1a.local". I0512 07:42:28.024269 8889 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0512 07:42:28.024299 8889 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0512 07:42:28.024322 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.025435 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.025458 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.030681 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.030732 8889 instancegroups.go:398] Draining the node: "master-1b.local". I0512 07:42:28.031573 8889 instancegroups.go:417] deleting node "master-1b.local" from kubernetes I0512 07:42:28.031609 8889 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0512 07:42:28.031632 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.032742 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.032762 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.037993 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.038021 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.038040 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.038052 8889 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0512 07:42:28.038995 8889 instancegroups.go:398] Draining the node: "node-1a.local". I0512 07:42:28.039917 8889 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0512 07:42:28.039960 8889 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0512 07:42:28.039985 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.041096 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.041119 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.046377 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.046426 8889 instancegroups.go:398] Draining the node: "node-1b.local". I0512 07:42:28.047421 8889 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0512 07:42:28.047454 8889 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0512 07:42:28.047478 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.048662 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.048697 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.053954 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.054010 8889 instancegroups.go:398] Draining the node: "node-1c.local". I0512 07:42:28.055036 8889 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0512 07:42:28.055071 8889 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0512 07:42:28.055094 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.056286 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.056344 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.061628 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.061660 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.061679 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.061693 8889 instancegroups.go:309] Tainting 3 nodes in "node-2" instancegroup. I0512 07:42:28.062838 8889 instancegroups.go:398] Draining the node: "node-2a.local". I0512 07:42:28.063631 8889 instancegroups.go:417] deleting node "node-2a.local" from kubernetes I0512 07:42:28.063664 8889 instancegroups.go:589] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0512 07:42:28.063688 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.064890 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.064927 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.070230 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.070280 8889 instancegroups.go:398] Draining the node: "node-2b.local". I0512 07:42:28.071118 8889 instancegroups.go:417] deleting node "node-2b.local" from kubernetes I0512 07:42:28.071187 8889 instancegroups.go:589] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0512 07:42:28.071215 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.072403 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.072437 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.077684 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.077733 8889 instancegroups.go:398] Draining the node: "node-2c.local". I0512 07:42:28.078594 8889 instancegroups.go:417] deleting node "node-2c.local" from kubernetes I0512 07:42:28.078629 8889 instancegroups.go:589] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0512 07:42:28.078651 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.079844 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.079879 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.085123 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.085156 8889 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdateWithForce (0.07s) === RUN TestRollingUpdateEmptyGroup I0512 07:42:28.085280 8889 rollingupdate.go:96] Cloud Instance Group length is zero. Not doing a rolling-update. --- PASS: TestRollingUpdateEmptyGroup (0.00s) === RUN TestRollingUpdateUnknownRole --- PASS: TestRollingUpdateUnknownRole (0.00s) === RUN TestRollingUpdateAllNeedUpdateFailsValidation I0512 07:42:28.085720 8889 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0512 07:42:28.085751 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.086947 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.086986 8889 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. E0512 07:42:28.087007 8889 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateFailsValidation (0.00s) === RUN TestRollingUpdateAllNeedUpdateErrorsValidation I0512 07:42:28.087315 8889 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0512 07:42:28.087344 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.088544 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.088580 8889 instancegroups.go:511] Cluster did not validate within deadline: testing validation error. E0512 07:42:28.088602 8889 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateFailsValidation I0512 07:42:28.088841 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.088870 8889 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. E0512 07:42:28.088885 8889 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateNodes1NeedsUpdateFailsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateErrorsValidation I0512 07:42:28.089032 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.089057 8889 instancegroups.go:511] Cluster did not validate within deadline: testing validation error. E0512 07:42:28.089072 8889 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateNodes1NeedsUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneMaster I0512 07:42:28.089325 8889 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0512 07:42:28.089355 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.090553 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.090594 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.095891 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.095958 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.095985 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.096001 8889 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0512 07:42:28.096700 8889 instancegroups.go:398] Draining the node: "master-1a.local". I0512 07:42:28.097457 8889 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0512 07:42:28.097491 8889 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0512 07:42:28.097514 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.098706 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.098759 8889 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. E0512 07:42:28.098781 8889 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneMaster I0512 07:42:28.099074 8889 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0512 07:42:28.099105 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.100309 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.100349 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.105623 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.105689 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.105712 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.105726 8889 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0512 07:42:28.106526 8889 instancegroups.go:398] Draining the node: "master-1a.local". I0512 07:42:28.107278 8889 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0512 07:42:28.107311 8889 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0512 07:42:28.107335 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.108533 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.108576 8889 instancegroups.go:511] Cluster did not validate within deadline: testing validation error. E0512 07:42:28.108598 8889 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateNonRelatedInstanceGroupFailure I0512 07:42:28.108889 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.108917 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.108931 8889 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0512 07:42:28.109917 8889 instancegroups.go:398] Draining the node: "node-1a.local". I0512 07:42:28.110672 8889 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0512 07:42:28.110705 8889 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0512 07:42:28.110728 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.111929 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.111966 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.117216 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.117274 8889 instancegroups.go:398] Draining the node: "node-1b.local". I0512 07:42:28.118107 8889 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0512 07:42:28.118142 8889 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0512 07:42:28.118165 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.119363 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.119399 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.124653 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.124703 8889 instancegroups.go:398] Draining the node: "node-1c.local". I0512 07:42:28.125565 8889 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0512 07:42:28.125599 8889 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0512 07:42:28.125623 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.126823 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.126858 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.132124 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.132158 8889 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNonRelatedInstanceGroupFailure (0.02s) === RUN TestRollingUpdateRelatedInstanceGroupFailure I0512 07:42:28.132444 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.132475 8889 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. E0512 07:42:28.132493 8889 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateRelatedInstanceGroupFailure (0.00s) === RUN TestRollingUpdateMasterGroupFailure I0512 07:42:28.132721 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.132749 8889 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. E0512 07:42:28.132765 8889 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateMasterGroupFailure (0.00s) === RUN TestRollingUpdateValidationErrorInstanceGroupNil I0512 07:42:28.132985 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.133013 8889 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. E0512 07:42:28.133027 8889 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateValidationErrorInstanceGroupNil (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneNode I0512 07:42:28.133213 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.133243 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.133256 8889 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0512 07:42:28.134235 8889 instancegroups.go:398] Draining the node: "node-1a.local". I0512 07:42:28.135016 8889 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0512 07:42:28.135053 8889 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0512 07:42:28.135076 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.136284 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.136329 8889 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. E0512 07:42:28.136350 8889 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneNode I0512 07:42:28.136556 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.136586 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.136600 8889 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0512 07:42:28.137554 8889 instancegroups.go:398] Draining the node: "node-1a.local". I0512 07:42:28.138331 8889 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0512 07:42:28.138365 8889 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0512 07:42:28.138388 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.139599 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.139639 8889 instancegroups.go:511] Cluster did not validate within deadline: testing validation error. E0512 07:42:28.139661 8889 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateFlappingValidation I0512 07:42:28.139909 8889 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0512 07:42:28.139938 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.141135 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.141206 8889 instancegroups.go:524] Cluster did not pass validation, will retry in "1ms": testing failure. I0512 07:42:28.142366 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.147648 8889 instancegroups.go:524] Cluster did not pass validation, will retry in "1ms": testing failure. I0512 07:42:28.148917 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.154159 8889 instancegroups.go:524] Cluster did not pass validation, will retry in "1ms": testing failure. I0512 07:42:28.155350 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.160608 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.160676 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.160701 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.160715 8889 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0512 07:42:28.161447 8889 instancegroups.go:398] Draining the node: "master-1a.local". I0512 07:42:28.162229 8889 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0512 07:42:28.162262 8889 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0512 07:42:28.162286 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.163490 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.163532 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.168786 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.168838 8889 instancegroups.go:398] Draining the node: "master-1b.local". I0512 07:42:28.169660 8889 instancegroups.go:417] deleting node "master-1b.local" from kubernetes I0512 07:42:28.169694 8889 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0512 07:42:28.169718 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.171274 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.171324 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.176605 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.176644 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.176671 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.176685 8889 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0512 07:42:28.177650 8889 instancegroups.go:398] Draining the node: "node-1a.local". I0512 07:42:28.178433 8889 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0512 07:42:28.178468 8889 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0512 07:42:28.178491 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.179685 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.179733 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.185006 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.185071 8889 instancegroups.go:398] Draining the node: "node-1b.local". I0512 07:42:28.186784 8889 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0512 07:42:28.186818 8889 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0512 07:42:28.186841 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.188019 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.188069 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.193379 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.193437 8889 instancegroups.go:398] Draining the node: "node-1c.local". I0512 07:42:28.194223 8889 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0512 07:42:28.194257 8889 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0512 07:42:28.194280 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.195476 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.195525 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.200828 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.200860 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.200885 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.200900 8889 instancegroups.go:309] Tainting 3 nodes in "node-2" instancegroup. I0512 07:42:28.201903 8889 instancegroups.go:398] Draining the node: "node-2a.local". I0512 07:42:28.202627 8889 instancegroups.go:417] deleting node "node-2a.local" from kubernetes I0512 07:42:28.202659 8889 instancegroups.go:589] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0512 07:42:28.202682 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.203871 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.203918 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.209205 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.209269 8889 instancegroups.go:398] Draining the node: "node-2b.local". I0512 07:42:28.210142 8889 instancegroups.go:417] deleting node "node-2b.local" from kubernetes I0512 07:42:28.210176 8889 instancegroups.go:589] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0512 07:42:28.210198 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.211377 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.211427 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.216720 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.216778 8889 instancegroups.go:398] Draining the node: "node-2c.local". I0512 07:42:28.217555 8889 instancegroups.go:417] deleting node "node-2c.local" from kubernetes I0512 07:42:28.217588 8889 instancegroups.go:589] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0512 07:42:28.217610 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.218816 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.218865 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.224165 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.224200 8889 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateFlappingValidation (0.08s) === RUN TestRollingUpdateValidatesAfterBastion I0512 07:42:28.224458 8889 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0512 07:42:28.224506 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.225717 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.225759 8889 instancegroups.go:524] Cluster did not pass validation, will retry in "1ms": testing failure. I0512 07:42:28.226919 8889 instancegroups.go:524] Cluster did not pass validation, will retry in "1ms": testing failure. I0512 07:42:28.228065 8889 instancegroups.go:524] Cluster did not pass validation, will retry in "1ms": testing failure. I0512 07:42:28.229248 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.234527 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.234596 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.234628 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.234645 8889 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0512 07:42:28.235403 8889 instancegroups.go:398] Draining the node: "master-1a.local". I0512 07:42:28.236220 8889 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0512 07:42:28.236254 8889 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0512 07:42:28.236281 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.237469 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.237504 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.242770 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.242822 8889 instancegroups.go:398] Draining the node: "master-1b.local". I0512 07:42:28.243635 8889 instancegroups.go:417] deleting node "master-1b.local" from kubernetes I0512 07:42:28.243668 8889 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0512 07:42:28.243692 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.244877 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.244914 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.250169 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.250209 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.250230 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.250244 8889 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0512 07:42:28.251228 8889 instancegroups.go:398] Draining the node: "node-1a.local". I0512 07:42:28.251975 8889 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0512 07:42:28.252011 8889 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0512 07:42:28.252036 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.253240 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.253276 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.258553 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.258605 8889 instancegroups.go:398] Draining the node: "node-1b.local". I0512 07:42:28.259415 8889 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0512 07:42:28.259448 8889 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0512 07:42:28.259471 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.260649 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.260684 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.265922 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.265974 8889 instancegroups.go:398] Draining the node: "node-1c.local". I0512 07:42:28.266817 8889 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0512 07:42:28.266851 8889 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0512 07:42:28.266873 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.268049 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.268085 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.273370 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.273402 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.273423 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.273438 8889 instancegroups.go:309] Tainting 3 nodes in "node-2" instancegroup. I0512 07:42:28.274357 8889 instancegroups.go:398] Draining the node: "node-2a.local". I0512 07:42:28.275122 8889 instancegroups.go:417] deleting node "node-2a.local" from kubernetes I0512 07:42:28.275156 8889 instancegroups.go:589] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0512 07:42:28.275179 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.276390 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.276433 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.281707 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.281773 8889 instancegroups.go:398] Draining the node: "node-2b.local". I0512 07:42:28.282667 8889 instancegroups.go:417] deleting node "node-2b.local" from kubernetes I0512 07:42:28.282700 8889 instancegroups.go:589] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0512 07:42:28.282723 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.283895 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.283930 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.289523 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.289578 8889 instancegroups.go:398] Draining the node: "node-2c.local". I0512 07:42:28.290636 8889 instancegroups.go:417] deleting node "node-2c.local" from kubernetes I0512 07:42:28.290685 8889 instancegroups.go:589] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0512 07:42:28.290722 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.291930 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.291974 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.297377 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.297412 8889 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateValidatesAfterBastion (0.07s) === RUN TestAddAnnotatedNodesToNeedsUpdate --- PASS: TestAddAnnotatedNodesToNeedsUpdate (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateCloudonly --- PASS: TestAddAnnotatedNodesToNeedsUpdateCloudonly (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateNodesMissing --- PASS: TestAddAnnotatedNodesToNeedsUpdateNodesMissing (0.00s) === RUN TestRollingUpdateTaintAllButOneNeedUpdate I0512 07:42:28.298044 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.298071 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.298084 8889 instancegroups.go:309] Tainting 2 nodes in "node-1" instancegroup. I0512 07:42:28.298780 8889 instancegroups.go:398] Draining the node: "node-1a.local". I0512 07:42:28.299594 8889 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0512 07:42:28.299627 8889 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0512 07:42:28.299648 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.300849 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.300884 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.306158 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.306208 8889 instancegroups.go:398] Draining the node: "node-1b.local". I0512 07:42:28.310505 8889 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0512 07:42:28.310546 8889 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0512 07:42:28.310571 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.311785 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.311822 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.317090 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.317123 8889 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateTaintAllButOneNeedUpdate (0.02s) === RUN TestRollingUpdateMaxSurgeIgnoredForMaster I0512 07:42:28.317412 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.317442 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.317455 8889 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0512 07:42:28.318250 8889 instancegroups.go:398] Draining the node: "master-1a.local". I0512 07:42:28.319008 8889 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0512 07:42:28.319041 8889 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0512 07:42:28.319064 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.320275 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.320312 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.325577 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.325626 8889 instancegroups.go:398] Draining the node: "master-1b.local". I0512 07:42:28.326508 8889 instancegroups.go:417] deleting node "master-1b.local" from kubernetes I0512 07:42:28.326541 8889 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0512 07:42:28.326564 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.327752 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.327796 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.333133 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.333167 8889 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeIgnoredForMaster (0.02s) === RUN TestRollingUpdateDisabled I0512 07:42:28.333466 8889 instancegroups.go:183] Rolling updates for InstanceGroup bastion-1 are disabled W0512 07:42:28.333508 8889 instancegroups.go:466] Not validating cluster as cloudonly flag is set. I0512 07:42:28.333528 8889 instancegroups.go:183] Rolling updates for InstanceGroup master-1 are disabled W0512 07:42:28.333542 8889 instancegroups.go:466] Not validating cluster as cloudonly flag is set. I0512 07:42:28.333560 8889 instancegroups.go:183] Rolling updates for InstanceGroup node-1 are disabled W0512 07:42:28.333569 8889 instancegroups.go:466] Not validating cluster as cloudonly flag is set. I0512 07:42:28.333585 8889 instancegroups.go:183] Rolling updates for InstanceGroup node-2 are disabled I0512 07:42:28.333599 8889 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabled (0.00s) === RUN TestRollingUpdateDisabledSurge I0512 07:42:28.333857 8889 instancegroups.go:568] Detaching instance "bastion-1a", in group "bastion-1". I0512 07:42:28.333910 8889 instancegroups.go:170] waiting for 1ms after detaching instance I0512 07:42:28.335093 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.335128 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.340369 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.340404 8889 instancegroups.go:183] Rolling updates for InstanceGroup bastion-1 are disabled I0512 07:42:28.340455 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.340475 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.340488 8889 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0512 07:42:28.341199 8889 instancegroups.go:183] Rolling updates for InstanceGroup master-1 are disabled I0512 07:42:28.341229 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.341249 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.341261 8889 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0512 07:42:28.342168 8889 instancegroups.go:566] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0512 07:42:28.342226 8889 instancegroups.go:170] waiting for 1ms after detaching instance I0512 07:42:28.343415 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.343449 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.348690 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.348724 8889 instancegroups.go:183] Rolling updates for InstanceGroup node-1 are disabled I0512 07:42:28.348739 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.348757 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.348771 8889 instancegroups.go:309] Tainting 3 nodes in "node-2" instancegroup. I0512 07:42:28.349682 8889 instancegroups.go:566] Detaching instance "node-2c", node "node-2c.local", in group "node-2". I0512 07:42:28.349738 8889 instancegroups.go:170] waiting for 1ms after detaching instance I0512 07:42:28.350938 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.350975 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:28.356218 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.356252 8889 instancegroups.go:183] Rolling updates for InstanceGroup node-2 are disabled I0512 07:42:28.356269 8889 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabledSurge (0.02s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdate I0512 07:42:28.356515 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.356552 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.356566 8889 instancegroups.go:309] Tainting 7 nodes in "node-1" instancegroup. I0512 07:42:28.358615 8889 instancegroups.go:398] Draining the node: "node-1a.local". I0512 07:42:28.359366 8889 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0512 07:42:28.359400 8889 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0512 07:42:28.359432 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.360641 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.360686 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.360730 8889 instancegroups.go:398] Draining the node: "node-1c.local". I0512 07:42:28.360935 8889 instancegroups.go:398] Draining the node: "node-1b.local". I0512 07:42:28.362306 8889 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0512 07:42:28.362346 8889 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0512 07:42:28.362707 8889 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0512 07:42:28.362753 8889 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0512 07:42:28.362802 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.364016 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.364107 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.464762 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.464832 8889 instancegroups.go:398] Draining the node: "node-1e.local". I0512 07:42:28.464851 8889 instancegroups.go:398] Draining the node: "node-1d.local". I0512 07:42:28.466344 8889 instancegroups.go:417] deleting node "node-1e.local" from kubernetes I0512 07:42:28.466380 8889 instancegroups.go:589] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0512 07:42:28.466657 8889 instancegroups.go:417] deleting node "node-1d.local" from kubernetes I0512 07:42:28.466685 8889 instancegroups.go:589] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0512 07:42:28.466714 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.467892 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.467945 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.467992 8889 instancegroups.go:398] Draining the node: "node-1f.local". I0512 07:42:28.468862 8889 instancegroups.go:417] deleting node "node-1f.local" from kubernetes I0512 07:42:28.468898 8889 instancegroups.go:589] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0512 07:42:28.468933 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.470137 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.470203 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.570780 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.570868 8889 instancegroups.go:398] Draining the node: "node-1g.local". I0512 07:42:28.571925 8889 instancegroups.go:417] deleting node "node-1g.local" from kubernetes I0512 07:42:28.571962 8889 instancegroups.go:589] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0512 07:42:28.572004 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.573225 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.573275 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.573295 8889 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxUnavailableAllButOneNeedUpdate I0512 07:42:28.573556 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.573590 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.573604 8889 instancegroups.go:309] Tainting 6 nodes in "node-1" instancegroup. I0512 07:42:28.575487 8889 instancegroups.go:398] Draining the node: "node-1b.local". I0512 07:42:28.575552 8889 instancegroups.go:398] Draining the node: "node-1a.local". I0512 07:42:28.576838 8889 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0512 07:42:28.576874 8889 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0512 07:42:28.577195 8889 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0512 07:42:28.577225 8889 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0512 07:42:28.577255 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.578448 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.578520 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.679166 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.679278 8889 instancegroups.go:398] Draining the node: "node-1d.local". I0512 07:42:28.679320 8889 instancegroups.go:398] Draining the node: "node-1c.local". I0512 07:42:28.680875 8889 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0512 07:42:28.680914 8889 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0512 07:42:28.681307 8889 instancegroups.go:417] deleting node "node-1d.local" from kubernetes I0512 07:42:28.681355 8889 instancegroups.go:589] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0512 07:42:28.681402 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.682615 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.682660 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.682701 8889 instancegroups.go:398] Draining the node: "node-1e.local". I0512 07:42:28.683691 8889 instancegroups.go:417] deleting node "node-1e.local" from kubernetes I0512 07:42:28.683725 8889 instancegroups.go:589] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0512 07:42:28.683756 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.684957 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.685036 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.785657 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.785718 8889 instancegroups.go:398] Draining the node: "node-1f.local". I0512 07:42:28.786745 8889 instancegroups.go:417] deleting node "node-1f.local" from kubernetes I0512 07:42:28.786785 8889 instancegroups.go:589] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0512 07:42:28.786819 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.788008 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.788054 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.788072 8889 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllButOneNeedUpdate (0.21s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdateMaster I0512 07:42:28.788311 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.788345 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.788360 8889 instancegroups.go:309] Tainting 7 nodes in "master-1" instancegroup. I0512 07:42:28.790501 8889 instancegroups.go:398] Draining the node: "master-1a.local". I0512 07:42:28.791269 8889 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0512 07:42:28.791303 8889 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0512 07:42:28.791335 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.792542 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.792600 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.792664 8889 instancegroups.go:398] Draining the node: "master-1c.local". I0512 07:42:28.792735 8889 instancegroups.go:398] Draining the node: "master-1b.local". I0512 07:42:28.794326 8889 instancegroups.go:417] deleting node "master-1c.local" from kubernetes I0512 07:42:28.794374 8889 instancegroups.go:589] Stopping instance "master-1c", node "master-1c.local", in group "master-1" (this may take a while). I0512 07:42:28.794727 8889 instancegroups.go:417] deleting node "master-1b.local" from kubernetes I0512 07:42:28.794767 8889 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0512 07:42:28.794810 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.796265 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.796330 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.896877 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.896963 8889 instancegroups.go:398] Draining the node: "master-1e.local". I0512 07:42:28.897040 8889 instancegroups.go:398] Draining the node: "master-1d.local". I0512 07:42:28.898523 8889 instancegroups.go:417] deleting node "master-1e.local" from kubernetes I0512 07:42:28.898562 8889 instancegroups.go:589] Stopping instance "master-1e", node "master-1e.local", in group "master-1" (this may take a while). I0512 07:42:28.898896 8889 instancegroups.go:417] deleting node "master-1d.local" from kubernetes I0512 07:42:28.898937 8889 instancegroups.go:589] Stopping instance "master-1d", node "master-1d.local", in group "master-1" (this may take a while). I0512 07:42:28.898972 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.900164 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.900219 8889 instancegroups.go:501] Cluster validated. I0512 07:42:28.900264 8889 instancegroups.go:398] Draining the node: "master-1f.local". I0512 07:42:28.901111 8889 instancegroups.go:417] deleting node "master-1f.local" from kubernetes I0512 07:42:28.901174 8889 instancegroups.go:589] Stopping instance "master-1f", node "master-1f.local", in group "master-1" (this may take a while). I0512 07:42:28.901210 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:28.902426 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:28.902490 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.003052 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.003132 8889 instancegroups.go:398] Draining the node: "master-1g.local". I0512 07:42:29.004101 8889 instancegroups.go:417] deleting node "master-1g.local" from kubernetes I0512 07:42:29.004136 8889 instancegroups.go:589] Stopping instance "master-1g", node "master-1g.local", in group "master-1" (this may take a while). I0512 07:42:29.004174 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.005407 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.005480 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.005502 8889 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdateMaster (0.22s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdate I0512 07:42:29.005851 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.005887 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.005903 8889 instancegroups.go:309] Tainting 6 nodes in "node-1" instancegroup. I0512 07:42:29.007772 8889 instancegroups.go:566] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0512 07:42:29.007868 8889 instancegroups.go:170] waiting for 1ms after detaching instance I0512 07:42:29.009093 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.009142 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.009163 8889 instancegroups.go:566] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0512 07:42:29.009222 8889 instancegroups.go:170] waiting for 1ms after detaching instance I0512 07:42:29.010448 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.010495 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.010547 8889 instancegroups.go:398] Draining the node: "node-1b.local". I0512 07:42:29.010621 8889 instancegroups.go:398] Draining the node: "node-1a.local". I0512 07:42:29.011925 8889 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0512 07:42:29.011966 8889 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0512 07:42:29.012307 8889 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0512 07:42:29.012347 8889 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0512 07:42:29.012382 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.013616 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.013688 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.114250 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.114304 8889 instancegroups.go:398] Draining the node: "node-1d.local". I0512 07:42:29.114326 8889 instancegroups.go:398] Draining the node: "node-1c.local". I0512 07:42:29.115851 8889 instancegroups.go:417] deleting node "node-1d.local" from kubernetes I0512 07:42:29.115905 8889 instancegroups.go:589] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0512 07:42:29.116192 8889 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0512 07:42:29.116230 8889 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0512 07:42:29.116263 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.117463 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.117507 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.117546 8889 instancegroups.go:398] Draining the node: "node-1e.local". I0512 07:42:29.118395 8889 instancegroups.go:417] deleting node "node-1e.local" from kubernetes I0512 07:42:29.118426 8889 instancegroups.go:589] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0512 07:42:29.118461 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.119673 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.119734 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.220313 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.220382 8889 instancegroups.go:398] Draining the node: "node-1f.local". I0512 07:42:29.221312 8889 instancegroups.go:417] deleting node "node-1f.local" from kubernetes I0512 07:42:29.221347 8889 instancegroups.go:589] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0512 07:42:29.221387 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.222601 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.222647 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.222665 8889 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeAllButOneNeedUpdate I0512 07:42:29.222912 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.222944 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.222958 8889 instancegroups.go:309] Tainting 6 nodes in "node-1" instancegroup. I0512 07:42:29.224889 8889 instancegroups.go:566] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0512 07:42:29.224938 8889 instancegroups.go:566] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0512 07:42:29.224979 8889 instancegroups.go:170] waiting for 1ms after detaching instance I0512 07:42:29.226181 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.226228 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.226277 8889 instancegroups.go:398] Draining the node: "node-1b.local". I0512 07:42:29.226373 8889 instancegroups.go:398] Draining the node: "node-1a.local". I0512 07:42:29.227719 8889 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0512 07:42:29.227756 8889 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0512 07:42:29.228102 8889 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0512 07:42:29.228141 8889 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0512 07:42:29.228174 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.229384 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.229453 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.330072 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.330124 8889 instancegroups.go:398] Draining the node: "node-1d.local". I0512 07:42:29.330191 8889 instancegroups.go:398] Draining the node: "node-1c.local". I0512 07:42:29.331342 8889 instancegroups.go:417] deleting node "node-1d.local" from kubernetes I0512 07:42:29.331377 8889 instancegroups.go:589] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0512 07:42:29.331646 8889 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0512 07:42:29.331675 8889 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0512 07:42:29.331703 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.332906 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.332949 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.332989 8889 instancegroups.go:398] Draining the node: "node-1e.local". I0512 07:42:29.333806 8889 instancegroups.go:417] deleting node "node-1e.local" from kubernetes I0512 07:42:29.333840 8889 instancegroups.go:589] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0512 07:42:29.333874 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.335085 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.335145 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.436014 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.436085 8889 instancegroups.go:398] Draining the node: "node-1f.local". I0512 07:42:29.437058 8889 instancegroups.go:417] deleting node "node-1f.local" from kubernetes I0512 07:42:29.437092 8889 instancegroups.go:589] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0512 07:42:29.437133 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.438332 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.438379 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.438397 8889 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllButOneNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeGreaterThanNeedUpdate I0512 07:42:29.438642 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.438668 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.438681 8889 instancegroups.go:309] Tainting 2 nodes in "node-1" instancegroup. I0512 07:42:29.439365 8889 instancegroups.go:566] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0512 07:42:29.439398 8889 instancegroups.go:566] Detaching instance "node-1a", node "node-1a.local", in group "node-1". I0512 07:42:29.439433 8889 instancegroups.go:170] waiting for 1ms after detaching instance I0512 07:42:29.440639 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.440674 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:29.445926 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.445982 8889 instancegroups.go:398] Draining the node: "node-1b.local". I0512 07:42:29.446006 8889 instancegroups.go:398] Draining the node: "node-1a.local". I0512 07:42:29.447521 8889 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0512 07:42:29.447889 8889 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0512 07:42:29.447928 8889 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0512 07:42:29.447967 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.447985 8889 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0512 07:42:29.448026 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.449144 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.449178 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:29.454452 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.454504 8889 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeGreaterThanNeedUpdate (0.02s) === RUN TestRollingUpdateDetachFails I0512 07:42:29.454734 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.454762 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.454775 8889 instancegroups.go:309] Tainting 2 nodes in "node-1" instancegroup. I0512 07:42:29.455453 8889 instancegroups.go:566] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0512 07:42:29.455494 8889 instancegroups.go:566] Detaching instance "node-1a", node "node-1a.local", in group "node-1". I0512 07:42:29.455533 8889 instancegroups.go:170] waiting for 1ms after detaching instance I0512 07:42:29.456724 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.456759 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:29.462018 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.462071 8889 instancegroups.go:398] Draining the node: "node-1b.local". I0512 07:42:29.462092 8889 instancegroups.go:398] Draining the node: "node-1a.local". I0512 07:42:29.463391 8889 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0512 07:42:29.463439 8889 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0512 07:42:29.463473 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.463778 8889 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0512 07:42:29.463819 8889 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0512 07:42:29.463844 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.464684 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.464719 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:29.470175 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.470209 8889 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDetachFails (0.02s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached I0512 07:42:29.470437 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.470469 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.470483 8889 instancegroups.go:309] Tainting 4 nodes in "node-1" instancegroup. I0512 07:42:29.471753 8889 instancegroups.go:566] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0512 07:42:29.471811 8889 instancegroups.go:170] waiting for 1ms after detaching instance I0512 07:42:29.473016 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.473061 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.473075 8889 instancegroups.go:566] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0512 07:42:29.473130 8889 instancegroups.go:170] waiting for 1ms after detaching instance I0512 07:42:29.474327 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.474378 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.474433 8889 instancegroups.go:398] Draining the node: "node-1c.local". I0512 07:42:29.474515 8889 instancegroups.go:398] Draining the node: "node-1a.local". I0512 07:42:29.474572 8889 instancegroups.go:398] Draining the node: "node-1b.local". I0512 07:42:29.476121 8889 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0512 07:42:29.476158 8889 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0512 07:42:29.476204 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.476517 8889 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0512 07:42:29.476553 8889 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0512 07:42:29.476588 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.476898 8889 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0512 07:42:29.476935 8889 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0512 07:42:29.476968 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.477419 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.578007 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.578060 8889 instancegroups.go:398] Draining the node: "node-1d.local". I0512 07:42:29.578909 8889 instancegroups.go:417] deleting node "node-1d.local" from kubernetes I0512 07:42:29.578941 8889 instancegroups.go:589] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0512 07:42:29.578974 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.580353 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.580434 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.580457 8889 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached (0.11s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached I0512 07:42:29.580794 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.580831 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.580846 8889 instancegroups.go:309] Tainting 7 nodes in "node-1" instancegroup. I0512 07:42:29.583005 8889 instancegroups.go:398] Draining the node: "node-1a.local". I0512 07:42:29.583833 8889 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0512 07:42:29.583874 8889 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0512 07:42:29.583909 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.585131 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.585179 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.585222 8889 instancegroups.go:398] Draining the node: "node-1e.local". I0512 07:42:29.585292 8889 instancegroups.go:398] Draining the node: "node-1c.local". I0512 07:42:29.586533 8889 instancegroups.go:417] deleting node "node-1e.local" from kubernetes I0512 07:42:29.586568 8889 instancegroups.go:589] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0512 07:42:29.586841 8889 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0512 07:42:29.586870 8889 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0512 07:42:29.586900 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.588099 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.588178 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.689128 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.689209 8889 instancegroups.go:398] Draining the node: "node-1g.local". I0512 07:42:29.689305 8889 instancegroups.go:398] Draining the node: "node-1f.local". I0512 07:42:29.691905 8889 instancegroups.go:417] deleting node "node-1g.local" from kubernetes I0512 07:42:29.692747 8889 instancegroups.go:417] deleting node "node-1f.local" from kubernetes I0512 07:42:29.692788 8889 instancegroups.go:589] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0512 07:42:29.692841 8889 instancegroups.go:589] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0512 07:42:29.692902 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.694123 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.694171 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.694217 8889 instancegroups.go:398] Draining the node: "node-1b.local". I0512 07:42:29.695074 8889 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0512 07:42:29.695105 8889 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0512 07:42:29.695135 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.696373 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.696433 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.797464 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.797531 8889 instancegroups.go:398] Draining the node: "node-1d.local". I0512 07:42:29.798425 8889 instancegroups.go:417] deleting node "node-1d.local" from kubernetes I0512 07:42:29.798457 8889 instancegroups.go:589] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0512 07:42:29.798488 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.799713 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.799758 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.799776 8889 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached (0.22s) === RUN TestRollingUpdateOnlyWarmPoolNodes I0512 07:42:29.800027 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.800054 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.800073 8889 instancegroups.go:131] deleting warm pool instance "node-1-wp-a" I0512 07:42:29.800094 8889 instancegroups.go:131] deleting warm pool instance "node-1-wp-b" I0512 07:42:29.800112 8889 instancegroups.go:131] deleting warm pool instance "node-1-wp-c" I0512 07:42:29.800129 8889 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateOnlyWarmPoolNodes (0.00s) === RUN TestRollingWarmPoolBeforeJoinedNodes I0512 07:42:29.800286 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.800310 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.800323 8889 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0512 07:42:29.801303 8889 instancegroups.go:131] deleting warm pool instance "node-1-wp-a" I0512 07:42:29.801337 8889 instancegroups.go:131] deleting warm pool instance "node-1-wp-b" I0512 07:42:29.801355 8889 instancegroups.go:131] deleting warm pool instance "node-1-wp-c" I0512 07:42:29.801394 8889 instancegroups.go:398] Draining the node: "node-1a.local". I0512 07:42:29.802136 8889 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0512 07:42:29.802169 8889 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0512 07:42:29.802191 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.803404 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.803437 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:29.808713 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.808764 8889 instancegroups.go:398] Draining the node: "node-1b.local". I0512 07:42:29.809586 8889 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0512 07:42:29.809618 8889 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0512 07:42:29.809641 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.810854 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.810888 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:29.816159 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.816210 8889 instancegroups.go:398] Draining the node: "node-1c.local". I0512 07:42:29.817030 8889 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0512 07:42:29.817062 8889 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0512 07:42:29.817085 8889 instancegroups.go:435] waiting for 1ms after terminating instance I0512 07:42:29.818286 8889 instancegroups.go:468] Validating the cluster. I0512 07:42:29.818327 8889 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0512 07:42:29.823621 8889 instancegroups.go:501] Cluster validated. I0512 07:42:29.823654 8889 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingWarmPoolBeforeJoinedNodes (0.02s) === RUN TestSettings === RUN TestSettings/DrainAndTerminate === RUN TestSettings/MaxUnavailable === RUN TestSettings/MaxSurge --- PASS: TestSettings (0.00s) --- PASS: TestSettings/DrainAndTerminate (0.00s) --- PASS: TestSettings/MaxUnavailable (0.00s) --- PASS: TestSettings/MaxSurge (0.00s) === RUN TestMaxUnavailable === RUN TestMaxUnavailable/0_1 === RUN TestMaxUnavailable/0%_1 === RUN TestMaxUnavailable/39%_10 === RUN TestMaxUnavailable/100%_10 --- PASS: TestMaxUnavailable (0.00s) --- PASS: TestMaxUnavailable/0_1 (0.00s) --- PASS: TestMaxUnavailable/0%_1 (0.00s) --- PASS: TestMaxUnavailable/39%_10 (0.00s) --- PASS: TestMaxUnavailable/100%_10 (0.00s) === RUN TestMaxSurge === RUN TestMaxSurge/0_1 === RUN TestMaxSurge/0%_1 === RUN TestMaxSurge/31%_10 === RUN TestMaxSurge/100%_10 --- PASS: TestMaxSurge (0.00s) --- PASS: TestMaxSurge/0_1 (0.00s) --- PASS: TestMaxSurge/0%_1 (0.00s) --- PASS: TestMaxSurge/31%_10 (0.00s) --- PASS: TestMaxSurge/100%_10 (0.00s) === RUN TestAWSDefault --- PASS: TestAWSDefault (0.00s) PASS ok k8s.io/kops/pkg/instancegroups 1.981s === RUN TestWriteToken === RUN TestWriteToken/WriteJson === RUN TestWriteToken/WriteJsonWithIndent --- PASS: TestWriteToken (0.00s) --- PASS: TestWriteToken/WriteJson (0.00s) --- PASS: TestWriteToken/WriteJsonWithIndent (0.00s) PASS ok k8s.io/kops/pkg/jsonutils 0.018s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.00s) PASS ok k8s.io/kops/pkg/k8scodecs 0.052s === RUN TestParse E0512 07:41:58.668293 4771 versions.go:96] unable to parse Kubernetes version "" E0512 07:41:58.668360 4771 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: TestParse (0.00s) === RUN TestIsGTE === RUN TestIsGTE/KV_greater_than_Version === RUN TestIsGTE/KV_greater_than_Version#01 === RUN TestIsGTE/KV_equal_Version === RUN TestIsGTE/KV_equal_Version#01 === RUN TestIsGTE/Version_greater_than_KV === RUN TestIsGTE/Version_greater_than_KV#01 --- PASS: TestIsGTE (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version#01 (0.00s) --- PASS: TestIsGTE/KV_equal_Version (0.00s) --- PASS: TestIsGTE/KV_equal_Version#01 (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV#01 (0.00s) PASS ok k8s.io/kops/pkg/k8sversion 0.035s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.01s) === RUN TestToVersionedJSON --- PASS: TestToVersionedJSON (0.00s) === RUN TestRewriteAPIGroup --- PASS: TestRewriteAPIGroup (0.00s) PASS ok k8s.io/kops/pkg/kopscodecs 0.069s === RUN TestBuildKubecfg === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster === RUN TestBuildKubecfg/Public_DNS_with_kops_auth_plugin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port --- PASS: TestBuildKubecfg (0.16s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster (0.01s) --- PASS: TestBuildKubecfg/Public_DNS_with_kops_auth_plugin (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port (0.03s) PASS ok k8s.io/kops/pkg/kubeconfig 0.310s ? k8s.io/kops/pkg/kubemanifest [no test files] === RUN Test_ProxyFunc --- PASS: Test_ProxyFunc (0.00s) === RUN TestBootstrapUserData --- PASS: TestBootstrapUserData (0.02s) === RUN TestValidateAWSVolumeAllow50ratio --- PASS: TestValidateAWSVolumeAllow50ratio (0.00s) PASS ok k8s.io/kops/pkg/model 0.070s ? k8s.io/kops/pkg/model/alimodel [no test files] === RUN TestRootVolumeOptimizationFlag --- PASS: TestRootVolumeOptimizationFlag (0.00s) === RUN TestAPIServerAdditionalSecurityGroupsWithNLB === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Master === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Node --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Master (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Node (0.00s) === RUN TestJoinSuffixes --- PASS: TestJoinSuffixes (0.00s) === RUN TestIAMServiceEC2 --- PASS: TestIAMServiceEC2 (0.00s) === RUN Test_formatAWSIAMStatement === RUN Test_formatAWSIAMStatement/namespace_and_name_without_wildcard === RUN Test_formatAWSIAMStatement/name_contains_wildcard === RUN Test_formatAWSIAMStatement/namespace_contains_wildcard --- PASS: Test_formatAWSIAMStatement (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_and_name_without_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/name_contains_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_contains_wildcard (0.00s) PASS ok k8s.io/kops/pkg/model/awsmodel 0.059s === RUN TestAPILoadBalancerModelBuilder_Build --- PASS: TestAPILoadBalancerModelBuilder_Build (0.00s) === RUN TestSubnetForLoadbalancer --- PASS: TestSubnetForLoadbalancer (0.00s) === RUN TestCloudTagsForInstanceGroup --- PASS: TestCloudTagsForInstanceGroup (0.00s) === RUN TestNetworkModelBuilder_Build --- PASS: TestNetworkModelBuilder_Build (0.00s) === RUN TestResourceGroupModelBuilder_Build --- PASS: TestResourceGroupModelBuilder_Build (0.00s) === RUN TestVMScaleSetModelBuilder_Build --- PASS: TestVMScaleSetModelBuilder_Build (0.00s) === RUN TestGetCapacity === RUN TestGetCapacity/test_case_0 === RUN TestGetCapacity/test_case_1 === RUN TestGetCapacity/test_case_2 === RUN TestGetCapacity/test_case_3 --- PASS: TestGetCapacity (0.00s) --- PASS: TestGetCapacity/test_case_0 (0.00s) --- PASS: TestGetCapacity/test_case_1 (0.00s) --- PASS: TestGetCapacity/test_case_2 (0.00s) --- PASS: TestGetCapacity/test_case_3 (0.00s) === RUN TestGetStorageProfile === RUN TestGetStorageProfile/test_case_0 === RUN TestGetStorageProfile/test_case_1 --- PASS: TestGetStorageProfile (0.00s) --- PASS: TestGetStorageProfile/test_case_0 (0.00s) --- PASS: TestGetStorageProfile/test_case_1 (0.00s) === RUN TestParseImage === RUN TestParseImage/test_case_0 === RUN TestParseImage/test_case_1 === RUN TestParseImage/test_case_2 === RUN TestParseImage/test_case_3 --- PASS: TestParseImage (0.00s) --- PASS: TestParseImage/test_case_0 (0.00s) --- PASS: TestParseImage/test_case_1 (0.00s) --- PASS: TestParseImage/test_case_2 (0.00s) --- PASS: TestParseImage/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/model/azuremodel 0.093s === RUN TestCloudConfigurationOptionsBuilder === RUN TestCloudConfigurationOptionsBuilder/neither === RUN TestCloudConfigurationOptionsBuilder/all_false === RUN TestCloudConfigurationOptionsBuilder/all_true === RUN TestCloudConfigurationOptionsBuilder/os_false === RUN TestCloudConfigurationOptionsBuilder/os_true === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_true === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_true --- PASS: TestCloudConfigurationOptionsBuilder (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/neither (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_true (0.00s) === RUN Test_Build_Containerd_Supported_Version --- PASS: Test_Build_Containerd_Supported_Version (0.00s) === RUN Test_Build_Containerd_Unneeded_Runtime --- PASS: Test_Build_Containerd_Unneeded_Runtime (0.00s) === RUN Test_Build_Containerd_Needed_Runtime --- PASS: Test_Build_Containerd_Needed_Runtime (0.00s) === RUN TestImage --- PASS: TestImage (0.00s) === RUN Test_Build_KCM_Builder --- PASS: Test_Build_KCM_Builder (0.00s) === RUN Test_Build_KCM_Builder_Change_Duration --- PASS: Test_Build_KCM_Builder_Change_Duration (0.00s) === RUN Test_Build_KCM_Builder_CIDR_Mask_Size === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: (0.00s) === RUN TestFeatureGatesKubernetesVersion --- PASS: TestFeatureGatesKubernetesVersion (0.00s) === RUN TestFeatureGatesOverride --- PASS: TestFeatureGatesOverride (0.00s) === RUN Test_Build_Scheduler_Without_PolicyConfigMap --- PASS: Test_Build_Scheduler_Without_PolicyConfigMap (0.00s) === RUN Test_Build_Scheduler_PolicyConfigMap_Supported_Version --- PASS: Test_Build_Scheduler_PolicyConfigMap_Supported_Version (0.00s) PASS ok k8s.io/kops/pkg/model/components 0.122s ? k8s.io/kops/pkg/model/components/addonmanifests [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awscloudcontrollermanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsebscsidriver [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsloadbalancercontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/clusterautoscaler [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/dnscontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/externaldns [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/nodeterminationhandler [no test files] === RUN Test_RunEtcdManagerBuilder === RUN Test_RunEtcdManagerBuilder/basedir=tests/minimal === RUN Test_RunEtcdManagerBuilder/basedir=tests/pollinterval === RUN Test_RunEtcdManagerBuilder/basedir=tests/proxy === RUN Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings W0512 07:42:11.655242 8363 model.go:347] overriding log level in manifest (embedded etcd manifest), new level is 3 W0512 07:42:11.655359 8363 model.go:511] overloading ENV var in manifest (embedded etcd manifest) with ETCD_QUOTA_BACKEND_BYTES=10737418240 W0512 07:42:11.656574 8363 model.go:347] overriding log level in manifest (embedded etcd manifest), new level is 3 W0512 07:42:11.656719 8363 model.go:511] overloading ENV var in manifest (embedded etcd manifest) with ETCD_QUOTA_BACKEND_BYTES=10737418240 --- PASS: Test_RunEtcdManagerBuilder (0.04s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/minimal (0.02s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/pollinterval (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/proxy (0.00s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings (0.01s) PASS ok k8s.io/kops/pkg/model/components/etcdmanager 0.097s ? k8s.io/kops/pkg/model/components/kopscontroller [no test files] === RUN Test_RunKubeApiserverBuilder === RUN Test_RunKubeApiserverBuilder/basedir=tests/minimal --- PASS: Test_RunKubeApiserverBuilder (0.01s) --- PASS: Test_RunKubeApiserverBuilder/basedir=tests/minimal (0.01s) PASS ok k8s.io/kops/pkg/model/components/kubeapiserver 0.070s === RUN TestDefaultInstanceGroupVolumeSize --- PASS: TestDefaultInstanceGroupVolumeSize (0.00s) PASS ok k8s.io/kops/pkg/model/defaults 0.065s ? k8s.io/kops/pkg/model/domodel [no test files] ? k8s.io/kops/pkg/model/gcemodel [no test files] === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) === RUN TestPolicyGeneration --- PASS: TestPolicyGeneration (0.00s) === RUN TestEmptyPolicy --- PASS: TestEmptyPolicy (0.00s) PASS ok k8s.io/kops/pkg/model/iam 0.055s === RUN TestServerGroupBuilder --- PASS: TestServerGroupBuilder (0.37s) PASS ok k8s.io/kops/pkg/model/openstackmodel 0.428s === RUN Test_NodeUpTabs --- PASS: Test_NodeUpTabs (0.00s) PASS ok k8s.io/kops/pkg/model/resources 0.069s ? k8s.io/kops/pkg/nodeidentity [no test files] ? k8s.io/kops/pkg/nodeidentity/aws [no test files] === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetVMSSNameFromProviderID === RUN TestGetVMSSNameFromProviderID/test_case_0 === RUN TestGetVMSSNameFromProviderID/test_case_1 === RUN TestGetVMSSNameFromProviderID/test_case_2 --- PASS: TestGetVMSSNameFromProviderID (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_0 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_1 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_2 (0.00s) === RUN TestIdentifyNode === RUN TestIdentifyNode/test_case_0 === RUN TestIdentifyNode/test_case_1 --- PASS: TestIdentifyNode (0.00s) --- PASS: TestIdentifyNode/test_case_0 (0.00s) --- PASS: TestIdentifyNode/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/nodeidentity/azure 0.049s ? k8s.io/kops/pkg/nodeidentity/do [no test files] ? k8s.io/kops/pkg/nodeidentity/gce [no test files] ? k8s.io/kops/pkg/nodeidentity/openstack [no test files] === RUN TestBuildNodeLabels === RUN TestBuildNodeLabels/RoleControlPlane === RUN TestBuildNodeLabels/RoleNode --- PASS: TestBuildNodeLabels (0.00s) --- PASS: TestBuildNodeLabels/RoleControlPlane (0.00s) --- PASS: TestBuildNodeLabels/RoleNode (0.00s) PASS ok k8s.io/kops/pkg/nodelabels 0.147s === RUN TestGenerateCertificate === RUN TestGenerateCertificate/selfsigned === RUN TestGenerateCertificate/client === RUN TestGenerateCertificate/server --- PASS: TestGenerateCertificate (0.01s) --- PASS: TestGenerateCertificate/selfsigned (0.00s) --- PASS: TestGenerateCertificate/client (0.00s) --- PASS: TestGenerateCertificate/server (0.00s) === RUN TestCertificateRoundTrip --- PASS: TestCertificateRoundTrip (0.00s) === RUN TestIssueCert === RUN TestIssueCert/ca === RUN TestIssueCert/client === RUN TestIssueCert/clientOneYear === RUN TestIssueCert/#00 === RUN TestIssueCert/server --- PASS: TestIssueCert (2.11s) --- PASS: TestIssueCert/ca (0.64s) --- PASS: TestIssueCert/client (0.50s) --- PASS: TestIssueCert/clientOneYear (0.98s) --- PASS: TestIssueCert/#00 (0.00s) --- PASS: TestIssueCert/server (0.00s) === RUN TestPrivateKeyRoundTrip --- PASS: TestPrivateKeyRoundTrip (0.00s) === RUN Test_AWSFingerprint_RsaKey1 --- PASS: Test_AWSFingerprint_RsaKey1 (0.00s) === RUN Test_AWSFingerprint_RsaKeyEncrypted --- PASS: Test_AWSFingerprint_RsaKeyEncrypted (0.00s) === RUN Test_AWSFingerprint_Ed25519Key --- PASS: Test_AWSFingerprint_Ed25519Key (0.00s) === RUN Test_AWSFingerprint_TrickyWhitespace --- PASS: Test_AWSFingerprint_TrickyWhitespace (0.00s) === RUN Test_AWSFingerprint_DsaKey --- PASS: Test_AWSFingerprint_DsaKey (0.00s) === RUN Test_OpenSSHFingerprint_RsaKey1 --- PASS: Test_OpenSSHFingerprint_RsaKey1 (0.00s) PASS ok k8s.io/kops/pkg/pki 2.126s === RUN TestLongDesc --- PASS: TestLongDesc (0.00s) PASS ok k8s.io/kops/pkg/pretty 0.015s ? k8s.io/kops/pkg/rbac [no test files] ? k8s.io/kops/pkg/resolver [no test files] ? k8s.io/kops/pkg/resources [no test files] ? k8s.io/kops/pkg/resources/ali [no test files] === RUN TestAddUntaggedRouteTables I0512 07:42:12.352413 8478 routetable.go:53] DescribeRouteTables: { } W0512 07:42:12.352539 8478 tags.go:67] cluster tag not found on route-table:rtb-1234 I0512 07:42:12.352549 8478 aws.go:224] Skipping route table in VPC, but with wrong cluster tag ("other.example.com") --- PASS: TestAddUntaggedRouteTables (0.00s) === RUN TestListRouteTables I0512 07:42:12.352644 8478 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["me.example.com"] }] } I0512 07:42:12.352733 8478 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/me.example.com"] }] } --- PASS: TestListRouteTables (0.00s) === RUN TestSharedVolume I0512 07:42:12.352798 8478 volumes.go:33] CreateVolume: { TagSpecifications: [{ ResourceType: "volume", Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "shared" }] }] } I0512 07:42:12.352886 8478 volumes.go:33] CreateVolume: { TagSpecifications: [{ Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "owned" }] }] } I0512 07:42:12.352907 8478 volumes.go:128] DescribeVolumes: { } W0512 07:42:12.352933 8478 aws.go:2236] (new) cluster tag not found on volume:vol-2 aws_test.go:192: 2 --- PASS: TestSharedVolume (0.00s) === RUN TestMatchesElbTags --- PASS: TestMatchesElbTags (0.00s) === RUN TestListVPCs I0512 07:42:12.353206 8478 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:12.353238 8478 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:12.353289 8478 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:12.353333 8478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:12.353372 8478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:12.353391 8478 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:12.353420 8478 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:12.353444 8478 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:12.353458 8478 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:12.353475 8478 tags.go:42] CreateTags { Resources: ["vpc-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "legacy.example.com" }] } I0512 07:42:12.353503 8478 tags.go:42] CreateTags { Resources: ["vpc-shared"], Tags: [{ Key: "kubernetes.io/cluster/shared.example.com", Value: "shared" }] } I0512 07:42:12.353521 8478 tags.go:42] CreateTags { Resources: ["vpc-shared-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "shared-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/shared-with-legacy.example.com", Value: "shared" }] } I0512 07:42:12.353558 8478 tags.go:42] CreateTags { Resources: ["vpc-owned"], Tags: [{ Key: "kubernetes.io/cluster/owned.example.com", Value: "owned" }] } I0512 07:42:12.353576 8478 tags.go:42] CreateTags { Resources: ["vpc-owned-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "owned-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/owned-with-legacy.example.com", Value: "owned" }] } I0512 07:42:12.353598 8478 tags.go:42] CreateTags { Resources: ["vpc-other"], Tags: [{ Key: "KubernetesCluster", Value: "other.example.com" },{ Key: "kubernetes.io/cluster/other.example.com", Value: "shared" }] } I0512 07:42:12.353630 8478 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["mismatch.example.com"] }] } I0512 07:42:12.353659 8478 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/mismatch.example.com"] }] } I0512 07:42:12.353677 8478 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["legacy.example.com"] }] } I0512 07:42:12.353694 8478 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/legacy.example.com"] }] } I0512 07:42:12.353714 8478 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared-with-legacy.example.com"] }] } I0512 07:42:12.353731 8478 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared-with-legacy.example.com"] }] } I0512 07:42:12.353750 8478 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared.example.com"] }] } I0512 07:42:12.353764 8478 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared.example.com"] }] } I0512 07:42:12.353782 8478 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned-with-legacy.example.com"] }] } I0512 07:42:12.353798 8478 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned-with-legacy.example.com"] }] } I0512 07:42:12.353816 8478 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned.example.com"] }] } I0512 07:42:12.353830 8478 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned.example.com"] }] } --- PASS: TestListVPCs (0.00s) PASS ok k8s.io/kops/pkg/resources/aws 0.052s === RUN TestListResourcesAzure --- PASS: TestListResourcesAzure (0.00s) === RUN TestIsOwnedByCluster === RUN TestIsOwnedByCluster/test_case_0 === RUN TestIsOwnedByCluster/test_case_1 === RUN TestIsOwnedByCluster/test_case_2 === RUN TestIsOwnedByCluster/test_case_3 --- PASS: TestIsOwnedByCluster (0.00s) --- PASS: TestIsOwnedByCluster/test_case_0 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_1 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_2 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/resources/azure 0.065s ? k8s.io/kops/pkg/resources/digitalocean [no test files] === RUN TestNameMatch --- PASS: TestNameMatch (0.00s) === RUN TestMatchesClusterNameWithUUID --- PASS: TestMatchesClusterNameWithUUID (0.00s) PASS ok k8s.io/kops/pkg/resources/gce 0.086s ? k8s.io/kops/pkg/resources/openstack [no test files] ? k8s.io/kops/pkg/resources/ops [no test files] ? k8s.io/kops/pkg/resources/spotinst [no test files] ? k8s.io/kops/pkg/sshcredentials [no test files] === RUN TestEscapeCommand --- PASS: TestEscapeCommand (0.00s) === RUN TestRawManifest --- PASS: TestRawManifest (0.00s) === RUN TestRawMixedManifest --- PASS: TestRawMixedManifest (0.00s) === RUN TestKeyPairOnlyManifest --- PASS: TestKeyPairOnlyManifest (0.00s) === RUN TestUnitFileExtensionValid === RUN TestUnitFileExtensionValid/valid_extension:_.automount === RUN TestUnitFileExtensionValid/valid_extension:_.device === RUN TestUnitFileExtensionValid/valid_extension:_.mount === RUN TestUnitFileExtensionValid/valid_extension:_.path === RUN TestUnitFileExtensionValid/valid_extension:_.scope === RUN TestUnitFileExtensionValid/valid_extension:_.service === RUN TestUnitFileExtensionValid/valid_extension:_.slice === RUN TestUnitFileExtensionValid/valid_extension:_.socket === RUN TestUnitFileExtensionValid/valid_extension:_.swap === RUN TestUnitFileExtensionValid/valid_extension:_.target === RUN TestUnitFileExtensionValid/valid_extension:_.timer === RUN TestUnitFileExtensionValid/invalid_extension:_(no_extension) === RUN TestUnitFileExtensionValid/invalid_extension:_.not-valid --- PASS: TestUnitFileExtensionValid (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.automount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.device (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.mount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.path (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.scope (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.service (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.slice (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.socket (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.swap (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.target (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.timer (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_(no_extension) (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_.not-valid (0.00s) PASS ok k8s.io/kops/pkg/systemd 0.017s ? k8s.io/kops/pkg/templates [no test files] ? k8s.io/kops/pkg/testutils [no test files] ? k8s.io/kops/pkg/testutils/golden [no test files] ? k8s.io/kops/pkg/tokens [no test files] ? k8s.io/kops/pkg/try [no test files] === RUN TestJoin --- PASS: TestJoin (0.00s) PASS ok k8s.io/kops/pkg/urls 0.026s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) PASS ok k8s.io/kops/pkg/util/stringorslice 0.014s === RUN Test_Overlap --- PASS: Test_Overlap (0.00s) === RUN Test_BelongsTo --- PASS: Test_BelongsTo (0.00s) === RUN Test_SplitInto8 --- PASS: Test_SplitInto8 (0.00s) PASS ok k8s.io/kops/pkg/util/subnet 0.034s === RUN TestRenderGeneralOK --- PASS: TestRenderGeneralOK (0.01s) === RUN TestRenderMissingValue --- PASS: TestRenderMissingValue (0.00s) === RUN TestRenderIndent --- PASS: TestRenderIndent (0.00s) === RUN TestRenderChannelFunctions --- PASS: TestRenderChannelFunctions (0.01s) === RUN TestRenderSnippet --- PASS: TestRenderSnippet (0.00s) === RUN TestRenderContext --- PASS: TestRenderContext (0.00s) === RUN TestAllowForMissingVars --- PASS: TestAllowForMissingVars (0.00s) === RUN TestRenderIntegration --- PASS: TestRenderIntegration (0.00s) PASS ok k8s.io/kops/pkg/util/templater 0.155s === RUN Test_ValidateCloudGroupMissing --- PASS: Test_ValidateCloudGroupMissing (0.00s) === RUN Test_ValidateNodesNotEnough --- PASS: Test_ValidateNodesNotEnough (0.00s) === RUN Test_ValidateDetachedNodesDontCount --- PASS: Test_ValidateDetachedNodesDontCount (0.00s) === RUN Test_ValidateNodeNotReady --- PASS: Test_ValidateNodeNotReady (0.00s) === RUN Test_ValidateMastersNotEnough --- PASS: Test_ValidateMastersNotEnough (0.00s) === RUN Test_ValidateMasterNotReady --- PASS: Test_ValidateMasterNotReady (0.00s) === RUN Test_ValidateMasterStaticPods --- PASS: Test_ValidateMasterStaticPods (0.00s) === RUN Test_ValidateNoPodFailures --- PASS: Test_ValidateNoPodFailures (0.00s) === RUN Test_ValidatePodFailure === RUN Test_ValidatePodFailure/pending-node-kube-system === RUN Test_ValidatePodFailure/pending-node-kube-system#01 === RUN Test_ValidatePodFailure/pending-node-kube-system#02 === RUN Test_ValidatePodFailure/pending-node-otherNamespace === RUN Test_ValidatePodFailure/pending-node-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-node-otherNamespace#02 === RUN Test_ValidatePodFailure/pending-cluster-kube-system === RUN Test_ValidatePodFailure/pending-cluster-kube-system#01 === RUN Test_ValidatePodFailure/pending-cluster-kube-system#02 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-node-kube-system === RUN Test_ValidatePodFailure/notready-node-kube-system#01 === RUN Test_ValidatePodFailure/notready-node-kube-system#02 === RUN Test_ValidatePodFailure/notready-node-otherNamespace === RUN Test_ValidatePodFailure/notready-node-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-node-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-cluster-kube-system === RUN Test_ValidatePodFailure/notready-cluster-kube-system#01 === RUN Test_ValidatePodFailure/notready-cluster-kube-system#02 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-node-kube-system === RUN Test_ValidatePodFailure/unknown-node-kube-system#01 === RUN Test_ValidatePodFailure/unknown-node-kube-system#02 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#01 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#02 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 --- PASS: Test_ValidatePodFailure (0.01s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 (0.00s) === RUN Test_ValidateBastionNodes === RUN Test_ValidateBastionNodes/instancegroup's_nodes_not_ready === RUN Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready --- PASS: Test_ValidateBastionNodes (0.00s) --- PASS: Test_ValidateBastionNodes/instancegroup's_nodes_not_ready (0.00s) --- PASS: Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready (0.00s) === RUN Test_ValidateDetachedNodesNotValidated --- PASS: Test_ValidateDetachedNodesNotValidated (0.00s) PASS ok k8s.io/kops/pkg/validation 0.062s ? k8s.io/kops/pkg/values [no test files] ? k8s.io/kops/pkg/wellknownoperators [no test files] ? k8s.io/kops/pkg/wellknownports [no test files] ? k8s.io/kops/pkg/wellknownusers [no test files] ? k8s.io/kops/pkg/zones [no test files] ? k8s.io/kops/protokube/cmd/protokube [no test files] ? k8s.io/kops/protokube/pkg/etcd [no test files] ? k8s.io/kops/protokube/pkg/gossip [no test files] ? k8s.io/kops/protokube/pkg/gossip/ali [no test files] ? k8s.io/kops/protokube/pkg/gossip/aws [no test files] === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetInternalIP --- PASS: TestGetInternalIP (0.00s) === RUN TestGetSeeds --- PASS: TestGetSeeds (0.00s) PASS ok k8s.io/kops/protokube/pkg/gossip/azure 0.037s ? k8s.io/kops/protokube/pkg/gossip/dns [no test files] === RUN TestRemovesDuplicateGuardedBlocks W0512 07:42:02.908203 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.908286 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.908297 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.908306 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.908321 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.908330 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.908338 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.908347 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.908357 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.908366 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.908374 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.908383 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.908391 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.908419 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.908432 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.908441 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.910841 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.910871 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.910945 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.910961 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.911030 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.911046 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.911117 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.911135 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.911213 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.911226 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.911306 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.911322 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.911378 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.911391 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.911471 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.911484 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.911538 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.911550 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.911604 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.911616 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.911683 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.911695 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.911755 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.911771 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.911833 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.911859 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.911921 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.911933 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.911988 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.912005 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.912075 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.912091 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.912142 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.912154 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.912207 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.912218 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.912278 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.912293 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.912357 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.912369 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.912418 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.912430 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.912499 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.912512 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.912574 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.912587 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.912650 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.912662 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.912724 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.912749 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.912802 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.912813 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.912876 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.912888 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.912942 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.912954 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.913009 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.913021 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.913088 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.913101 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.913159 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.913172 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.913239 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.913260 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.913319 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.913331 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.913384 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.913395 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.913454 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.913469 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.913541 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.913553 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.913634 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.913647 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.913706 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.913740 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.913803 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.913816 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.913877 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.913890 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.913949 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.913961 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.914012 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.914023 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.914089 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.914109 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.914166 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.914178 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.914233 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.914245 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.914301 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.914313 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.914370 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.914382 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.914443 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.914458 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.915431 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.915449 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.915518 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.915534 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.915598 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.915610 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.915665 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.915679 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.915738 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.915752 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.915814 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.915833 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.915893 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.915907 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.915970 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.915984 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.916044 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.916058 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.916123 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.916140 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.916203 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.916217 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.916273 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.916288 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.916344 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.916358 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.916424 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.916438 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.916491 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.916505 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.916575 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.916598 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.916654 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.916668 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.916726 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.916741 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.916809 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.916823 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.916885 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.916899 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.916956 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.916971 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.917033 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.917048 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.917104 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.917118 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.917177 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.917191 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.917253 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.917268 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.917336 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.917350 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.917407 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.917422 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.917477 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.917497 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.917552 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.917570 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.917631 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.918029 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.918122 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.918163 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.918265 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.918304 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.918380 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.918413 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.918502 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.918536 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.918610 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.918626 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.918682 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.918696 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.918755 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.918770 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.918833 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.918848 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.918913 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.918928 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.918986 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.919000 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.919059 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.919078 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.919146 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.919160 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.919220 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.919235 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.919295 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.919309 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.919360 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.919380 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.919441 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.919455 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.919515 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.919531 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.919594 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.919615 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.919667 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.919682 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.919736 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.919747 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.919797 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.919809 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRemovesDuplicateGuardedBlocks (0.01s) === RUN TestRecoversFromBadNesting W0512 07:42:02.920388 6416 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0512 07:42:02.920399 6416 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0512 07:42:02.920408 6416 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0512 07:42:02.920415 6416 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0512 07:42:02.920430 6416 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0512 07:42:02.920719 6416 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0512 07:42:02.920728 6416 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0512 07:42:02.920755 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.920765 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.920773 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.920781 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.920789 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.920798 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.920806 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.920814 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.920823 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.920836 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.920845 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.920853 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.920862 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.920871 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.920879 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.920889 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.929478 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.929536 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.929608 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.929620 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.929669 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.929680 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.929735 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.929747 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.929799 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.929810 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.929859 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.929878 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.929932 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.929943 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.930003 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.930015 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.930064 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.930074 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.930127 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.930138 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.930187 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.930198 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.930263 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.930274 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.930322 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.930333 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.930394 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.930405 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.930458 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.930469 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.930530 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.930541 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.930604 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.930615 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.930668 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.930679 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.930729 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.930747 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.930809 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.930820 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.930876 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.930887 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.930941 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.930958 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.931008 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.931019 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.931082 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.931093 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.931155 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.931168 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.931221 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.931232 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.931298 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.931311 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.931379 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.931391 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.931462 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.931475 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.931526 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.931537 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.931587 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.931604 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.931660 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.931672 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.931745 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.931757 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.931814 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.931825 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.931872 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.931892 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.931948 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.931958 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.932007 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.932023 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.932081 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.932096 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.932157 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.932175 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.932230 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.932242 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.932290 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.932302 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.932354 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.932366 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.932449 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.932462 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.932529 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.932542 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.932602 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.932615 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.932673 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.932686 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.932756 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.932769 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.932831 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.932844 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.932901 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.932913 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.932976 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.932988 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.933091 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.933103 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.933171 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.933188 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.933240 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.933252 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.933303 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.933321 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.933384 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.933396 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.933459 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.933470 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.933525 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.933537 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.933591 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.933603 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.933657 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.933669 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.933730 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.933742 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.933796 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.933808 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.933858 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.933869 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.933927 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.933940 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.934003 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.934015 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.934084 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.934096 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.934153 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.934165 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.934220 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.934232 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.934285 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.934297 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.934347 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.934359 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.934424 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.934444 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.934511 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.934524 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.934582 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.934594 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.934647 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.934665 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.934723 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.934743 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.934797 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.934809 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.934880 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.934892 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.934949 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.934961 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.935044 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.935057 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.935109 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.935121 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.935181 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.935193 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.935247 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.935258 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.935316 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.935328 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.935377 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.935392 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.935449 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.935461 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.935520 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.935532 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.935589 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.935606 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.935658 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.935669 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.935726 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.935738 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.935791 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.935803 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.935862 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.935875 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.935929 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.935941 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.936003 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.936015 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.936083 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.936096 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.936157 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.936169 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.936220 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.936232 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.936288 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.936300 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.936359 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.936372 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.936435 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.936446 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0512 07:42:02.936496 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0512 07:42:02.936508 6416 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRecoversFromBadNesting (0.02s) PASS ok k8s.io/kops/protokube/pkg/gossip/dns/hosts 0.057s ? k8s.io/kops/protokube/pkg/gossip/dns/provider [no test files] ? k8s.io/kops/protokube/pkg/gossip/do [no test files] ? k8s.io/kops/protokube/pkg/gossip/memberlist [no test files] ? k8s.io/kops/protokube/pkg/gossip/mesh [no test files] ? k8s.io/kops/protokube/pkg/gossip/openstack [no test files] ? k8s.io/kops/protokube/pkg/hostmount [no test files] ? k8s.io/kops/protokube/pkg/protokube [no test files] === RUN TestSerializeEmptyCluster --- PASS: TestSerializeEmptyCluster (0.01s) PASS ok k8s.io/kops/tests/codecs 0.098s ? k8s.io/kops/tests/fuzz [no test files] === RUN TestKopsUpgrades --- PASS: TestKopsUpgrades (0.01s) === RUN TestKubernetesUpgrades --- PASS: TestKubernetesUpgrades (0.00s) === RUN TestFindImage --- PASS: TestFindImage (0.00s) === RUN TestRecommendedKubernetesVersion --- PASS: TestRecommendedKubernetesVersion (0.00s) === RUN TestOrdering --- PASS: TestOrdering (0.00s) === RUN TestChannelsSelfConsistent --- PASS: TestChannelsSelfConsistent (0.04s) === RUN TestChannelImages === RUN TestChannelImages/stable-channel === RUN TestChannelImages/alpha-channel --- PASS: TestChannelImages (0.00s) --- PASS: TestChannelImages/stable-channel (0.00s) --- PASS: TestChannelImages/alpha-channel (0.00s) PASS ok k8s.io/kops/tests/integration/channel 0.134s ? k8s.io/kops/tests/integration/channel/simple [no test files] === RUN TestConversionMinimal === RUN TestConversionMinimal/legacy-v1alpha2-v1alpha2 --- PASS: TestConversionMinimal (0.02s) --- PASS: TestConversionMinimal/legacy-v1alpha2-v1alpha2 (0.02s) PASS ok k8s.io/kops/tests/integration/conversion 0.097s ? k8s.io/kops/upup/models [no test files] === RUN Test_tryResourceAsString --- PASS: Test_tryResourceAsString (0.00s) === RUN Test_DryrunTarget_PrintReport --- PASS: Test_DryrunTarget_PrintReport (0.00s) === RUN TestWriteFile I0512 07:42:07.761854 7246 files.go:57] Writing file "/tmp/fitest3565328053/SubDir/test1.tmp" I0512 07:42:07.762050 7246 files.go:113] Changing file mode for "/tmp/fitest3565328053/SubDir/test1.tmp" to -rw-r--r-- --- PASS: TestWriteFile (0.00s) === RUN TestBigInt_Format --- PASS: TestBigInt_Format (0.00s) === RUN TestVFSCAStoreRoundTrip --- PASS: TestVFSCAStoreRoundTrip (0.00s) === RUN TestVFSCAStoreRoundTripWithVault vfs_castore_test.go:161: No vault dev token set --- SKIP: TestVFSCAStoreRoundTripWithVault (0.00s) === RUN TestAddItem === RUN TestAddItem/first_primary === RUN TestAddItem/first_secondary === RUN TestAddItem/firstBigSerial_primary === RUN TestAddItem/firstBigSerial_secondary === RUN TestAddItem/after_primary === RUN TestAddItem/after_secondary === RUN TestAddItem/bigSerialAfter_primary === RUN TestAddItem/bigSerialAfter_secondary === RUN TestAddItem/before_primary === RUN TestAddItem/before_secondary === RUN TestAddItem/first_certonly_primary === RUN TestAddItem/first_certonly_secondary === RUN TestAddItem/after_certonly_primary === RUN TestAddItem/after_certonly_secondary === RUN TestAddItem/before_certonly_primary === RUN TestAddItem/before_certonly_secondary --- PASS: TestAddItem (0.00s) --- PASS: TestAddItem/first_primary (0.00s) --- PASS: TestAddItem/first_secondary (0.00s) --- PASS: TestAddItem/firstBigSerial_primary (0.00s) --- PASS: TestAddItem/firstBigSerial_secondary (0.00s) --- PASS: TestAddItem/after_primary (0.00s) --- PASS: TestAddItem/after_secondary (0.00s) --- PASS: TestAddItem/bigSerialAfter_primary (0.00s) --- PASS: TestAddItem/bigSerialAfter_secondary (0.00s) --- PASS: TestAddItem/before_primary (0.00s) --- PASS: TestAddItem/before_secondary (0.00s) --- PASS: TestAddItem/first_certonly_primary (0.00s) --- PASS: TestAddItem/first_certonly_secondary (0.00s) --- PASS: TestAddItem/after_certonly_primary (0.00s) --- PASS: TestAddItem/after_certonly_secondary (0.00s) --- PASS: TestAddItem/before_certonly_primary (0.00s) --- PASS: TestAddItem/before_certonly_secondary (0.00s) PASS ok k8s.io/kops/upup/pkg/fi 0.037s === RUN TestBootstrapChannelBuilder_BuildTasks I0512 07:42:16.091333 8676 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:16.091461 8676 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:16.091499 8676 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:16.091558 8676 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:16.091596 8676 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:16.091616 8676 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:16.091638 8676 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:16.091675 8676 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:16.091706 8676 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:16.098821 8676 zones.go:93] ListHostedZonesPages { } I0512 07:42:16.153411 8676 zones.go:93] ListHostedZonesPages { } I0512 07:42:16.225331 8676 zones.go:93] ListHostedZonesPages { } I0512 07:42:16.289956 8676 zones.go:93] ListHostedZonesPages { } I0512 07:42:16.361471 8676 zones.go:93] ListHostedZonesPages { } I0512 07:42:16.419723 8676 zones.go:93] ListHostedZonesPages { } I0512 07:42:16.483410 8676 zones.go:93] ListHostedZonesPages { } I0512 07:42:16.542496 8676 zones.go:93] ListHostedZonesPages { } I0512 07:42:16.643982 8676 zones.go:93] ListHostedZonesPages { } I0512 07:42:16.748133 8676 zones.go:93] ListHostedZonesPages { } I0512 07:42:17.756527 8676 zones.go:93] ListHostedZonesPages { } I0512 07:42:18.921077 8676 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_BuildTasks (2.88s) === RUN TestBootstrapChannelBuilder_ServiceAccountIAM I0512 07:42:18.976155 8676 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:18.976240 8676 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:18.976291 8676 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:18.976348 8676 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:18.976391 8676 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:18.976420 8676 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:18.976453 8676 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:18.976490 8676 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:18.976519 8676 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:18.976611 8676 featureflag.go:165] Unknown FeatureFlag "+UseServiceAccountExternalPermissions" I0512 07:42:18.978539 8676 zones.go:93] ListHostedZonesPages { } I0512 07:42:19.064023 8676 featureflag.go:165] Unknown FeatureFlag "-UseServiceAccountExternalPermissions" --- PASS: TestBootstrapChannelBuilder_ServiceAccountIAM (0.09s) === RUN TestBootstrapChannelBuilder_AWSCloudController I0512 07:42:19.064344 8676 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:19.064412 8676 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0512 07:42:19.064440 8676 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0512 07:42:19.064471 8676 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0512 07:42:19.064515 8676 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0512 07:42:19.064533 8676 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0512 07:42:19.064550 8676 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0512 07:42:19.064571 8676 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0512 07:42:19.064586 8676 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0512 07:42:19.065729 8676 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_AWSCloudController (0.13s) === RUN TestContainerdVersionUrlHash === RUN TestContainerdVersionUrlHash/1.3.4-amd64 === RUN TestContainerdVersionUrlHash/1.3.4-arm64 === RUN TestContainerdVersionUrlHash/1.3.10-amd64 === RUN TestContainerdVersionUrlHash/1.3.10-arm64 === RUN TestContainerdVersionUrlHash/1.4.1-amd64 === RUN TestContainerdVersionUrlHash/1.4.1-arm64 === RUN TestContainerdVersionUrlHash/1.4.4-amd64 === RUN TestContainerdVersionUrlHash/1.4.4-arm64 --- PASS: TestContainerdVersionUrlHash (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-arm64 (0.00s) === RUN TestContainerdVersionUrl === RUN TestContainerdVersionUrl/1.4.1- === RUN TestContainerdVersionUrl/1.4.1-arm === RUN TestContainerdVersionUrl/-amd64 === RUN TestContainerdVersionUrl/-arm64 === RUN TestContainerdVersionUrl/1.1.1-amd64 === RUN TestContainerdVersionUrl/1.1.1-arm64 === RUN TestContainerdVersionUrl/1.3.5-amd64 === RUN TestContainerdVersionUrl/1.3.5-arm64 === RUN TestContainerdVersionUrl/1.3.4-amd64 === RUN TestContainerdVersionUrl/1.3.4-arm64 === RUN TestContainerdVersionUrl/1.4.1-amd64 === RUN TestContainerdVersionUrl/1.4.1-arm64 === RUN TestContainerdVersionUrl/1.4.3-amd64 === RUN TestContainerdVersionUrl/1.4.3-arm64 --- PASS: TestContainerdVersionUrl (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1- (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionUrl/-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionHash === RUN TestContainerdVersionHash/1.4.1- === RUN TestContainerdVersionHash/1.4.1-arm === RUN TestContainerdVersionHash/-amd64 === RUN TestContainerdVersionHash/-arm64 === RUN TestContainerdVersionHash/1.1.1-amd64 === RUN TestContainerdVersionHash/1.1.1-arm64 === RUN TestContainerdVersionHash/1.3.5-amd64 === RUN TestContainerdVersionHash/1.3.5-arm64 === RUN TestContainerdVersionHash/1.4.1-amd64 === RUN TestContainerdVersionHash/1.4.1-arm64 === RUN TestContainerdVersionHash/1.4.3-amd64 === RUN TestContainerdVersionHash/1.4.3-arm64 --- PASS: TestContainerdVersionHash (0.00s) --- PASS: TestContainerdVersionHash/1.4.1- (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionHash/-amd64 (0.00s) --- PASS: TestContainerdVersionHash/-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionsHashesAmd64 containerd_test.go:316: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesAmd64 (0.00s) === RUN TestContainerdVersionsHashesArm64 containerd_test.go:331: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesArm64 (0.00s) === RUN TestDeepValidate_OK --- PASS: TestDeepValidate_OK (0.00s) === RUN TestDeepValidate_NoNodeZones --- PASS: TestDeepValidate_NoNodeZones (0.00s) === RUN TestDeepValidate_NoMasterZones --- PASS: TestDeepValidate_NoMasterZones (0.00s) === RUN TestDeepValidate_BadZone deepvalidate_test.go:57: Zone validation not checked by DeepValidate --- SKIP: TestDeepValidate_BadZone (0.00s) === RUN TestDeepValidate_MixedRegion deepvalidate_test.go:69: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_MixedRegion (0.00s) === RUN TestDeepValidate_RegionAsZone deepvalidate_test.go:83: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_RegionAsZone (0.00s) === RUN TestDeepValidate_NotIncludedZone --- PASS: TestDeepValidate_NotIncludedZone (0.00s) === RUN TestDeepValidate_DuplicateZones --- PASS: TestDeepValidate_DuplicateZones (0.00s) === RUN TestDeepValidate_ExtraMasterZone --- PASS: TestDeepValidate_ExtraMasterZone (0.00s) === RUN TestDeepValidate_EvenEtcdClusterSize --- PASS: TestDeepValidate_EvenEtcdClusterSize (0.00s) === RUN TestDeepValidate_MissingEtcdMember --- PASS: TestDeepValidate_MissingEtcdMember (0.00s) === RUN TestPopulateClusterSpec_Proxy --- PASS: TestPopulateClusterSpec_Proxy (0.00s) === RUN TestPrecreateDNSNames --- PASS: TestPrecreateDNSNames (0.00s) === RUN TestDockerVersionUrlHash === RUN TestDockerVersionUrlHash/19.03.13-amd64 === RUN TestDockerVersionUrlHash/19.03.13-arm64 === RUN TestDockerVersionUrlHash/18.06.3-amd64 === RUN TestDockerVersionUrlHash/18.06.3-arm64 === RUN TestDockerVersionUrlHash/17.03.1-amd64 === RUN TestDockerVersionUrlHash/17.03.1-arm64 --- PASS: TestDockerVersionUrlHash (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-arm64 (0.00s) === RUN TestDockerVersionUrl === RUN TestDockerVersionUrl/19.03.13- === RUN TestDockerVersionUrl/19.03.13-arm === RUN TestDockerVersionUrl/-amd64 === RUN TestDockerVersionUrl/-arm64 === RUN TestDockerVersionUrl/18.06.3-amd64 === RUN TestDockerVersionUrl/18.06.3-arm64 === RUN TestDockerVersionUrl/19.03.13-amd64 === RUN TestDockerVersionUrl/19.03.13-arm64 --- PASS: TestDockerVersionUrl (0.00s) --- PASS: TestDockerVersionUrl/19.03.13- (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm (0.00s) --- PASS: TestDockerVersionUrl/-amd64 (0.00s) --- PASS: TestDockerVersionUrl/-arm64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm64 (0.00s) === RUN TestDockerVersionHash === RUN TestDockerVersionHash/19.03.13- === RUN TestDockerVersionHash/19.03.13-arm === RUN TestDockerVersionHash/-amd64 === RUN TestDockerVersionHash/-arm64 === RUN TestDockerVersionHash/1.1.1-amd64 === RUN TestDockerVersionHash/1.1.1-arm64 === RUN TestDockerVersionHash/19.03.13-amd64 === RUN TestDockerVersionHash/19.03.13-arm64 --- PASS: TestDockerVersionHash (0.00s) --- PASS: TestDockerVersionHash/19.03.13- (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm (0.00s) --- PASS: TestDockerVersionHash/-amd64 (0.00s) --- PASS: TestDockerVersionHash/-arm64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm64 (0.00s) === RUN TestDockerVersionsHashesAmd64 docker_test.go:245: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesAmd64 (0.00s) === RUN TestDockerVersionsHashesArm64 docker_test.go:260: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesArm64 (0.00s) === RUN Test_FindCNIAssetFromEnvironmentVariable --- PASS: Test_FindCNIAssetFromEnvironmentVariable (0.00s) === RUN Test_FindCNIAssetFromDefaults118 --- PASS: Test_FindCNIAssetFromDefaults118 (0.18s) === RUN Test_FindCNIAssetFromDefaults122 --- PASS: Test_FindCNIAssetFromDefaults122 (0.12s) === RUN TestRemoveSharedPrefix --- PASS: TestRemoveSharedPrefix (0.00s) === RUN TestCreateEtcdCluster --- PASS: TestCreateEtcdCluster (0.00s) === RUN TestSetupNetworking W0512 07:42:19.523806 8676 new_cluster.go:935] flannel UDP mode is not recommended; consider flannel-vxlan instead --- PASS: TestSetupNetworking (0.00s) === RUN TestPopulateCluster_Default_NoError --- PASS: TestPopulateCluster_Default_NoError (0.00s) === RUN TestPopulateCluster_Subnets === RUN TestPopulateCluster_Subnets/100.64.0.0/10 === RUN TestPopulateCluster_Subnets/10.0.0.0/9 === RUN TestPopulateCluster_Subnets/10.0.0.0/8 === RUN TestPopulateCluster_Subnets/::/0 --- PASS: TestPopulateCluster_Subnets (0.01s) --- PASS: TestPopulateCluster_Subnets/100.64.0.0/10 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/9 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/8 (0.00s) --- PASS: TestPopulateCluster_Subnets/::/0 (0.00s) === RUN TestPopulateCluster_Docker_Spec --- PASS: TestPopulateCluster_Docker_Spec (0.00s) === RUN TestPopulateCluster_StorageDefault --- PASS: TestPopulateCluster_StorageDefault (0.00s) === RUN TestPopulateCluster_Kubenet --- PASS: TestPopulateCluster_Kubenet (0.00s) === RUN TestPopulateCluster_CNI --- PASS: TestPopulateCluster_CNI (0.00s) === RUN TestPopulateCluster_Custom_CIDR --- PASS: TestPopulateCluster_Custom_CIDR (0.00s) === RUN TestPopulateCluster_IsolateMasters --- PASS: TestPopulateCluster_IsolateMasters (0.00s) === RUN TestPopulateCluster_IsolateMastersFalse --- PASS: TestPopulateCluster_IsolateMastersFalse (0.00s) === RUN TestPopulateCluster_Name_Required --- PASS: TestPopulateCluster_Name_Required (0.00s) === RUN TestPopulateCluster_Zone_Required --- PASS: TestPopulateCluster_Zone_Required (0.00s) === RUN TestPopulateCluster_NetworkCIDR_Required --- PASS: TestPopulateCluster_NetworkCIDR_Required (0.00s) === RUN TestPopulateCluster_NonMasqueradeCIDR_Required --- PASS: TestPopulateCluster_NonMasqueradeCIDR_Required (0.00s) === RUN TestPopulateCluster_CloudProvider_Required --- PASS: TestPopulateCluster_CloudProvider_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidNil_Required --- PASS: TestPopulateCluster_TopologyInvalidNil_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidValue_Required --- PASS: TestPopulateCluster_TopologyInvalidValue_Required (0.00s) === RUN TestPopulateCluster_BastionInvalidMatchingValues_Required --- PASS: TestPopulateCluster_BastionInvalidMatchingValues_Required (0.00s) === RUN TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required --- PASS: TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required (0.00s) === RUN TestPopulateCluster_APIServerCount --- PASS: TestPopulateCluster_APIServerCount (0.00s) === RUN TestPopulateCluster_AnonymousAuth --- PASS: TestPopulateCluster_AnonymousAuth (0.00s) === RUN TestPopulateCluster_DockerVersion --- PASS: TestPopulateCluster_DockerVersion (0.00s) === RUN TestPopulateCluster_KubeController_High_Enough_Version --- PASS: TestPopulateCluster_KubeController_High_Enough_Version (0.00s) === RUN TestPopulateInstanceGroup_Name_Required --- PASS: TestPopulateInstanceGroup_Name_Required (0.00s) === RUN TestPopulateInstanceGroup_Role_Required --- PASS: TestPopulateInstanceGroup_Role_Required (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision --- PASS: TestPopulateInstanceGroup_AddTaintsCollision (0.00s) === RUN TestPopulateInstanceGroup_AddTaints --- PASS: TestPopulateInstanceGroup_AddTaints (0.00s) === RUN TestMachineArchitecture === RUN TestMachineArchitecture/t2.micro-amd64 === RUN TestMachineArchitecture/t3.micro-amd64 === RUN TestMachineArchitecture/a1.large-arm64 --- PASS: TestMachineArchitecture (0.00s) --- PASS: TestMachineArchitecture/t2.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/t3.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/a1.large-arm64 (0.00s) === RUN TestRuncVersionsHashesAmd64 runc_test.go:28: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesAmd64 (0.00s) === RUN TestRuncVersionsHashesArm64 runc_test.go:43: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesArm64 (0.00s) === RUN Test_AssignSubnets I0512 07:42:19.571051 8676 subnets.go:182] Assigned CIDR 10.32.0.0/11 to subnet a I0512 07:42:19.571078 8676 subnets.go:182] Assigned CIDR 10.32.0.0/11 to subnet a I0512 07:42:19.571090 8676 subnets.go:182] Assigned CIDR 10.64.0.0/11 to subnet b I0512 07:42:19.571108 8676 subnets.go:182] Assigned CIDR 10.32.0.0/11 to subnet b I0512 07:42:19.571119 8676 subnets.go:182] Assigned CIDR 10.64.0.0/11 to subnet a I0512 07:42:19.571147 8676 subnets.go:182] Assigned CIDR 10.32.0.0/11 to subnet b I0512 07:42:19.571181 8676 subnets.go:182] Assigned CIDR 10.160.0.0/11 to subnet b I0512 07:42:19.571212 8676 subnets.go:182] Assigned CIDR 10.32.0.0/11 to subnet a I0512 07:42:19.571227 8676 subnets.go:196] Assigned CIDR 10.0.0.0/14 to subnet a I0512 07:42:19.571248 8676 subnets.go:182] Assigned CIDR 10.32.0.0/11 to subnet a I0512 07:42:19.571260 8676 subnets.go:182] Assigned CIDR 10.64.0.0/11 to subnet b I0512 07:42:19.571271 8676 subnets.go:182] Assigned CIDR 10.96.0.0/11 to subnet b I0512 07:42:19.571281 8676 subnets.go:182] Assigned CIDR 10.128.0.0/11 to subnet c I0512 07:42:19.571291 8676 subnets.go:182] Assigned CIDR 10.160.0.0/11 to subnet c I0512 07:42:19.571301 8676 subnets.go:196] Assigned CIDR 10.0.0.0/14 to subnet a I0512 07:42:19.571312 8676 subnets.go:196] Assigned CIDR 10.4.0.0/14 to subnet b I0512 07:42:19.571323 8676 subnets.go:196] Assigned CIDR 10.8.0.0/14 to subnet c --- PASS: Test_AssignSubnets (0.00s) === RUN Test_TemplateFunctions_CloudControllerConfigArgv === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration (0.00s) === RUN Test_BuildMirroredAsset === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-amd64 === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-arm64 === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup --- PASS: Test_BuildMirroredAsset (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-amd64 (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-arm64 (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup (0.00s) === RUN TestValidateFull_Default_Validates --- PASS: TestValidateFull_Default_Validates (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_NoDot --- PASS: TestValidateFull_ClusterName_InvalidDNS_NoDot (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_Invalid --- PASS: TestValidateFull_ClusterName_InvalidDNS_Invalid (0.00s) === RUN TestValidateFull_ClusterName_Required --- PASS: TestValidateFull_ClusterName_Required (0.00s) === RUN TestValidateFull_UpdatePolicy_Valid === RUN TestValidateFull_UpdatePolicy_Valid/missing === RUN TestValidateFull_UpdatePolicy_Valid/automatic === RUN TestValidateFull_UpdatePolicy_Valid/external --- PASS: TestValidateFull_UpdatePolicy_Valid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/missing (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/automatic (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/external (0.00s) === RUN TestValidateFull_UpdatePolicy_Invalid === RUN TestValidateFull_UpdatePolicy_Invalid/empty === RUN TestValidateFull_UpdatePolicy_Invalid/populated --- PASS: TestValidateFull_UpdatePolicy_Invalid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/empty (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/populated (0.00s) === RUN Test_Validate_Kubenet_With_14 --- PASS: Test_Validate_Kubenet_With_14 (0.00s) === RUN TestValidate_ClusterName_Import --- PASS: TestValidate_ClusterName_Import (0.00s) === RUN TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity --- PASS: TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup 3.555s ? k8s.io/kops/upup/pkg/fi/cloudup/alitasks [no test files] === RUN TestFindRegion --- PASS: TestFindRegion (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/aliup 0.091s === RUN TestGetASGTagsToDelete --- PASS: TestGetASGTagsToDelete (0.00s) === RUN TestProcessCompare --- PASS: TestProcessCompare (0.00s) === RUN TestAutoscalingGroupTerraformRender I0512 07:42:14.927641 8628 target.go:112] Terraform output is in /tmp/kops-render-3719424199 I0512 07:42:14.929022 8628 target.go:112] Terraform output is in /tmp/kops-render-3719424199 --- PASS: TestAutoscalingGroupTerraformRender (0.00s) === RUN TestAutoscalingGroupCloudformationRender I0512 07:42:14.929824 8628 target.go:168] Cloudformation output is in /tmp/kops-render-2661026957 --- PASS: TestAutoscalingGroupCloudformationRender (0.00s) === RUN TestGetEBSVolumeTagsToDelete --- PASS: TestGetEBSVolumeTagsToDelete (0.00s) === RUN TestSharedEgressOnlyInternetGatewayDoesNotRename I0512 07:42:14.930354 8628 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0512 07:42:14.930425 8628 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0512 07:42:14.930472 8628 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { TagSpecifications: [{ ResourceType: "egress-only-internet-gateway", Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] }], VpcId: "vpc-1" } I0512 07:42:14.930634 8628 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0512 07:42:14.930698 8628 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:42:14.930751 8628 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0512 07:42:14.930766 8628 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0512 07:42:14.930875 8628 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0512 07:42:14.930902 8628 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0512 07:42:14.930931 8628 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0512 07:42:14.930983 8628 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0512 07:42:14.931047 8628 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0512 07:42:14.931190 8628 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0512 07:42:14.931231 8628 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:42:14.931276 8628 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0512 07:42:14.931287 8628 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0512 07:42:14.931311 8628 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0512 07:42:14.931347 8628 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0512 07:42:14.931388 8628 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedEgressOnlyInternetGatewayDoesNotRename (0.00s) === RUN TestElasticIPCreate I0512 07:42:14.931706 8628 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0512 07:42:14.931777 8628 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0512 07:42:14.931888 8628 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0512 07:42:14.931931 8628 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0512 07:42:14.931958 8628 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:42:14.931985 8628 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0512 07:42:14.932134 8628 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0512 07:42:14.932312 8628 subnets.go:105] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0512 07:42:14.932362 8628 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:42:14.932393 8628 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:42:14.932428 8628 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0512 07:42:14.932487 8628 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:42:14.932614 8628 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "eip1" }] }] } I0512 07:42:14.932664 8628 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:42:14.932683 8628 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedElasticIp", Value: "192.0.2.1" },{ Key: "AssociatedElasticIpAllocationId", Value: "eipalloc-1" }] } I0512 07:42:14.932711 8628 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0512 07:42:14.932846 8628 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0512 07:42:14.932888 8628 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0512 07:42:14.932925 8628 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0512 07:42:14.932938 8628 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0512 07:42:14.932957 8628 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:42:14.932975 8628 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0512 07:42:14.933003 8628 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0512 07:42:14.933056 8628 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:42:14.933074 8628 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0512 07:42:14.933119 8628 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:42:14.933158 8628 address.go:120] DescribeAddresses: { Filters: [{ Name: "public-ip", Values: ["192.0.2.1"] }] } I0512 07:42:14.933184 8628 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0512 07:42:14.933215 8628 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestElasticIPCreate (0.00s) === RUN TestSharedInternetGatewayDoesNotRename I0512 07:42:14.933347 8628 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0512 07:42:14.933376 8628 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0512 07:42:14.933392 8628 internetgateways.go:66] CreateInternetGateway: { } I0512 07:42:14.933415 8628 tags.go:42] CreateTags { Resources: ["igw-1"], Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] } I0512 07:42:14.933510 8628 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0512 07:42:14.933543 8628 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:42:14.933586 8628 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0512 07:42:14.933598 8628 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0512 07:42:14.933650 8628 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0512 07:42:14.933671 8628 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0512 07:42:14.933694 8628 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0512 07:42:14.933749 8628 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0512 07:42:14.933817 8628 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0512 07:42:14.933929 8628 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0512 07:42:14.933961 8628 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:42:14.933991 8628 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0512 07:42:14.934004 8628 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0512 07:42:14.934029 8628 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0512 07:42:14.934057 8628 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0512 07:42:14.934110 8628 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedInternetGatewayDoesNotRename (0.00s) === RUN TestLaunchTemplateCloudformationRender I0512 07:42:14.934741 8628 target.go:168] Cloudformation output is in /tmp/kops-render-3751308074 I0512 07:42:14.934954 8628 target.go:168] Cloudformation output is in /tmp/kops-render-3751308074 --- PASS: TestLaunchTemplateCloudformationRender (0.00s) === RUN TestLaunchTemplateTerraformRender I0512 07:42:14.937665 8628 target.go:112] Terraform output is in /tmp/kops-render-1452308759 I0512 07:42:14.940096 8628 target.go:112] Terraform output is in /tmp/kops-render-1452308759 --- PASS: TestLaunchTemplateTerraformRender (0.01s) === RUN TestParseRemovalRule --- PASS: TestParseRemovalRule (0.00s) === RUN TestPortRemovalRule --- PASS: TestPortRemovalRule (0.00s) === RUN TestPortRemovalRule_Zero --- PASS: TestPortRemovalRule_Zero (0.00s) === RUN TestSecurityGroupCreate I0512 07:42:14.940745 8628 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0512 07:42:14.940811 8628 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0512 07:42:14.940940 8628 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0512 07:42:14.940998 8628 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0512 07:42:14.941028 8628 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:42:14.941057 8628 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0512 07:42:14.941152 8628 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0512 07:42:14.941368 8628 securitygroups.go:40] CreateSecurityGroup: { Description: "Description", GroupName: "sg1", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "sg1" }] }], VpcId: "vpc-1" } I0512 07:42:14.941434 8628 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0512 07:42:14.941469 8628 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0512 07:42:14.941615 8628 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0512 07:42:14.941662 8628 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0512 07:42:14.941713 8628 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0512 07:42:14.941729 8628 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0512 07:42:14.941758 8628 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:42:14.941781 8628 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0512 07:42:14.941822 8628 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0512 07:42:14.941897 8628 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSecurityGroupCreate (0.00s) === RUN Test_Subnet_ValidateRequired --- PASS: Test_Subnet_ValidateRequired (0.00s) === RUN Test_Subnet_CannotChangeSubnet --- PASS: Test_Subnet_CannotChangeSubnet (0.00s) === RUN TestSubnetCreate I0512 07:42:14.942280 8628 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0512 07:42:14.942331 8628 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0512 07:42:14.942431 8628 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0512 07:42:14.942479 8628 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0512 07:42:14.942510 8628 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:42:14.942539 8628 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0512 07:42:14.942588 8628 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0512 07:42:14.942724 8628 subnets.go:105] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0512 07:42:14.942776 8628 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:42:14.942808 8628 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:42:14.942836 8628 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0512 07:42:14.942975 8628 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0512 07:42:14.943023 8628 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0512 07:42:14.943077 8628 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0512 07:42:14.943098 8628 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0512 07:42:14.943133 8628 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:42:14.943165 8628 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0512 07:42:14.943212 8628 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0512 07:42:14.943280 8628 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:42:14.943309 8628 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSubnetCreate (0.00s) === RUN TestSubnetCreateIPv6 I0512 07:42:14.943596 8628 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0512 07:42:14.943644 8628 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0512 07:42:14.943748 8628 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0512 07:42:14.943795 8628 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0512 07:42:14.943824 8628 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:42:14.943854 8628 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0512 07:42:14.943906 8628 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0512 07:42:14.944023 8628 subnets.go:105] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0512 07:42:14.944074 8628 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:42:14.944112 8628 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:42:14.944139 8628 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:42:14.944305 8628 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0512 07:42:14.944358 8628 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0512 07:42:14.944593 8628 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0512 07:42:14.944646 8628 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0512 07:42:14.944701 8628 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0512 07:42:14.944723 8628 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0512 07:42:14.944755 8628 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:42:14.944785 8628 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0512 07:42:14.944834 8628 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0512 07:42:14.944902 8628 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:42:14.944930 8628 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:42:14.944988 8628 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6 (0.00s) === RUN TestSubnetCreateIPv6NetNum I0512 07:42:14.945298 8628 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0512 07:42:14.945349 8628 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0512 07:42:14.945449 8628 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0512 07:42:14.945493 8628 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0512 07:42:14.945523 8628 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:42:14.945554 8628 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0512 07:42:14.945615 8628 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0512 07:42:14.945751 8628 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:42:14.945865 8628 subnets.go:105] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0512 07:42:14.945921 8628 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:42:14.945957 8628 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:42:14.945983 8628 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0512 07:42:14.946017 8628 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0512 07:42:14.946167 8628 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0512 07:42:14.946218 8628 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0512 07:42:14.946275 8628 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0512 07:42:14.946297 8628 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0512 07:42:14.946331 8628 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:42:14.946362 8628 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0512 07:42:14.946409 8628 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0512 07:42:14.946518 8628 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:42:14.946553 8628 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:42:14.946617 8628 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6NetNum (0.00s) === RUN TestSharedSubnetCreateDoesNotCreateNew I0512 07:42:14.946775 8628 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "ExistingVPC" }] }] } I0512 07:42:14.946857 8628 subnets.go:105] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "ExistingSubnet" }] }], VpcId: "vpc-1" } I0512 07:42:14.947016 8628 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0512 07:42:14.947068 8628 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:42:14.947117 8628 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0512 07:42:14.947140 8628 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0512 07:42:14.947226 8628 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0512 07:42:14.947261 8628 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0512 07:42:14.947306 8628 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0512 07:42:14.947350 8628 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:42:14.947463 8628 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0512 07:42:14.947495 8628 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0512 07:42:14.947536 8628 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0512 07:42:14.947693 8628 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0512 07:42:14.947767 8628 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:42:14.947843 8628 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0512 07:42:14.947879 8628 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0512 07:42:14.947923 8628 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0512 07:42:14.947982 8628 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0512 07:42:14.948052 8628 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedSubnetCreateDoesNotCreateNew (0.00s) === RUN TestVPCCreate I0512 07:42:14.948321 8628 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0512 07:42:14.948366 8628 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0512 07:42:14.948463 8628 vpcs.go:89] CreateVpc: { CidrBlock: "172.21.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0512 07:42:14.948504 8628 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0512 07:42:14.948532 8628 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:42:14.948560 8628 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0512 07:42:14.948678 8628 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0512 07:42:14.948725 8628 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0512 07:42:14.948775 8628 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0512 07:42:14.948792 8628 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0512 07:42:14.948821 8628 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0512 07:42:14.948846 8628 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestVPCCreate (0.00s) === RUN Test4758 --- PASS: Test4758 (0.00s) === RUN TestSharedVPCAdditionalCIDR I0512 07:42:14.949038 8628 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "172.22.0.0/16", VpcId: "vpc-1" } I0512 07:42:14.949178 8628 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0512 07:42:14.949224 8628 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc-1"] }] } I0512 07:42:14.949278 8628 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0512 07:42:14.949299 8628 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0512 07:42:14.949335 8628 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestSharedVPCAdditionalCIDR (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awstasks 0.067s === RUN TestValidateRegion --- PASS: TestValidateRegion (0.00s) === RUN TestFindRegion --- PASS: TestFindRegion (0.00s) === RUN TestEC2TagSpecification === RUN TestEC2TagSpecification/No_tags === RUN TestEC2TagSpecification/simple_tag --- PASS: TestEC2TagSpecification (0.00s) --- PASS: TestEC2TagSpecification/No_tags (0.00s) --- PASS: TestEC2TagSpecification/simple_tag (0.00s) === RUN Test_GetResourceName32 --- PASS: Test_GetResourceName32 (0.00s) === RUN TestTruncateString === RUN TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false --- PASS: TestTruncateString (0.00s) --- PASS: TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awsup 0.047s === RUN TestZoneToLocation === RUN TestZoneToLocation/test_case_0 === RUN TestZoneToLocation/test_case_1 === RUN TestZoneToLocation/test_case_2 --- PASS: TestZoneToLocation (0.00s) --- PASS: TestZoneToLocation/test_case_0 (0.00s) --- PASS: TestZoneToLocation/test_case_1 (0.00s) --- PASS: TestZoneToLocation/test_case_2 (0.00s) === RUN TestZoneToAvailabilityZoneNumber === RUN TestZoneToAvailabilityZoneNumber/test_case_0 === RUN TestZoneToAvailabilityZoneNumber/test_case_1 === RUN TestZoneToAvailabilityZoneNumber/test_case_2 --- PASS: TestZoneToAvailabilityZoneNumber (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_0 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_1 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_2 (0.00s) === RUN TestFindEtcdStatus --- PASS: TestFindEtcdStatus (0.00s) === RUN TestGetCloudGroups --- PASS: TestGetCloudGroups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azure 0.094s === RUN TestDiskRenderAzure I0512 07:42:09.022189 7543 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRenderAzure (0.00s) === RUN TestDiskFind --- PASS: TestDiskFind (0.00s) === RUN TestDiskRun I0512 07:42:09.022574 7543 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRun (0.00s) === RUN TestDiskCheckChanges === RUN TestDiskCheckChanges/test_case_0 === RUN TestDiskCheckChanges/test_case_1 === RUN TestDiskCheckChanges/test_case_2 === RUN TestDiskCheckChanges/test_case_3 --- PASS: TestDiskCheckChanges (0.00s) --- PASS: TestDiskCheckChanges/test_case_0 (0.00s) --- PASS: TestDiskCheckChanges/test_case_1 (0.00s) --- PASS: TestDiskCheckChanges/test_case_2 (0.00s) --- PASS: TestDiskCheckChanges/test_case_3 (0.00s) === RUN TestLoadBalancerRenderAzure I0512 07:42:09.022798 7543 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRenderAzure (0.00s) === RUN TestLoadBalancerFind loadbalancer_test.go:122: *azuretasks.LoadBalancer {"Name":"loadbalancer","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Subnet":{"Name":"subnet","Lifecycle":"","ResourceGroup":null,"VirtualNetwork":null,"CIDR":null,"Shared":null},"External":true,"Tags":null,"ForAPIServer":false} --- PASS: TestLoadBalancerFind (0.00s) === RUN TestLoadBalancerRun I0512 07:42:09.023309 7543 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRun (0.00s) === RUN TestLoadBalancerCheckChanges === RUN TestLoadBalancerCheckChanges/test_case_0 === RUN TestLoadBalancerCheckChanges/test_case_1 === RUN TestLoadBalancerCheckChanges/test_case_2 === RUN TestLoadBalancerCheckChanges/test_case_3 --- PASS: TestLoadBalancerCheckChanges (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_0 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_1 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_2 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_3 (0.00s) === RUN TestPublicIPAddressRenderAzure I0512 07:42:09.023519 7543 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRenderAzure (0.00s) === RUN TestPublicIPAddressFind publicipaddress_test.go:100: *azuretasks.PublicIPAddress {"Name":"publicIPAddress","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Tags":null} --- PASS: TestPublicIPAddressFind (0.00s) === RUN TestPublicIPAddressRun I0512 07:42:09.023859 7543 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRun (0.00s) === RUN TestPublicIPAddressCheckChanges === RUN TestPublicIPAddressCheckChanges/test_case_0 === RUN TestPublicIPAddressCheckChanges/test_case_1 === RUN TestPublicIPAddressCheckChanges/test_case_2 === RUN TestPublicIPAddressCheckChanges/test_case_3 --- PASS: TestPublicIPAddressCheckChanges (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_0 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_1 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_2 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_3 (0.00s) === RUN TestResourceGroupRenderAzure I0512 07:42:09.024114 7543 resourcegroup.go:103] Creating a new Resource Group with name: rg I0512 07:42:09.024128 7543 resourcegroup.go:105] Updating a Resource Group with name: rg --- PASS: TestResourceGroupRenderAzure (0.00s) === RUN TestResourceGroupFind --- PASS: TestResourceGroupFind (0.00s) === RUN TestResourceGroupRun I0512 07:42:09.024313 7543 resourcegroup.go:103] Creating a new Resource Group with name: rg --- PASS: TestResourceGroupRun (0.00s) === RUN TestResourceGroupCheckChanges === RUN TestResourceGroupCheckChanges/test_case_0 === RUN TestResourceGroupCheckChanges/test_case_1 === RUN TestResourceGroupCheckChanges/test_case_2 === RUN TestResourceGroupCheckChanges/test_case_3 --- PASS: TestResourceGroupCheckChanges (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_0 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_1 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_2 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_3 (0.00s) === RUN TestRoleAssignmentRenderAzure --- PASS: TestRoleAssignmentRenderAzure (0.00s) === RUN TestRoleAssignmentFind --- PASS: TestRoleAssignmentFind (0.00s) === RUN TestRoleAssignmentFind_NoPrincipalID --- PASS: TestRoleAssignmentFind_NoPrincipalID (0.00s) === RUN TestRoleAssignmentCheckChanges === RUN TestRoleAssignmentCheckChanges/test_case_0 === RUN TestRoleAssignmentCheckChanges/test_case_1 === RUN TestRoleAssignmentCheckChanges/test_case_2 === RUN TestRoleAssignmentCheckChanges/test_case_3 --- PASS: TestRoleAssignmentCheckChanges (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_0 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_1 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_2 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_3 (0.00s) === RUN TestSubnetRenderAzure I0512 07:42:09.024881 7543 subnet.go:106] Creating a new Subnet with name: vnet --- PASS: TestSubnetRenderAzure (0.00s) === RUN TestSubnetFind --- PASS: TestSubnetFind (0.00s) === RUN TestSubnetCheckChanges === RUN TestSubnetCheckChanges/test_case_0 === RUN TestSubnetCheckChanges/test_case_1 === RUN TestSubnetCheckChanges/test_case_2 === RUN TestSubnetCheckChanges/test_case_3 --- PASS: TestSubnetCheckChanges (0.00s) --- PASS: TestSubnetCheckChanges/test_case_0 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_1 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_2 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_3 (0.00s) === RUN TestVirtualNetworkRenderAzure I0512 07:42:09.025099 7543 virtualnetwork.go:116] Creating a new Virtual Network with name: vnet --- PASS: TestVirtualNetworkRenderAzure (0.00s) === RUN TestVirtualNetworkFind --- PASS: TestVirtualNetworkFind (0.00s) === RUN TestVirtualNetworkRun I0512 07:42:09.025319 7543 virtualnetwork.go:116] Creating a new Virtual Network with name: rg --- PASS: TestVirtualNetworkRun (0.00s) === RUN TestVirtualNetworkCheckChanges === RUN TestVirtualNetworkCheckChanges/test_case_0 === RUN TestVirtualNetworkCheckChanges/test_case_1 === RUN TestVirtualNetworkCheckChanges/test_case_2 === RUN TestVirtualNetworkCheckChanges/test_case_3 --- PASS: TestVirtualNetworkCheckChanges (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_0 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_1 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_2 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_3 (0.00s) === RUN TestSubnetIDParse --- PASS: TestSubnetIDParse (0.00s) === RUN TestLoadBalancerIDParse --- PASS: TestLoadBalancerIDParse (0.00s) === RUN TestVMScaleSetRenderAzure I0512 07:42:09.025630 7543 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRenderAzure (0.00s) === RUN TestVMScaleSetFind --- PASS: TestVMScaleSetFind (0.00s) === RUN TestVMScaleSetRun I0512 07:42:09.025967 7543 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRun (0.00s) === RUN TestVMScaleSetCheckChanges === RUN TestVMScaleSetCheckChanges/test_case_0 === RUN TestVMScaleSetCheckChanges/test_case_1 === RUN TestVMScaleSetCheckChanges/test_case_2 === RUN TestVMScaleSetCheckChanges/test_case_3 --- PASS: TestVMScaleSetCheckChanges (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_0 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_1 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_2 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_3 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azuretasks 0.085s ? k8s.io/kops/upup/pkg/fi/cloudup/bootstrapchannelbuilder [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/cloudformation [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/do [no test files] === RUN Test_Find === RUN Test_Find/successfully_found_volume === RUN Test_Find/no_volume_found === RUN Test_Find/error_from_server --- PASS: Test_Find (0.00s) --- PASS: Test_Find/successfully_found_volume (0.00s) --- PASS: Test_Find/no_volume_found (0.00s) --- PASS: Test_Find/error_from_server (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/dotasks 0.114s ? k8s.io/kops/upup/pkg/fi/cloudup/gce [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcediscovery [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcemetadata [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmsigner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmverifier [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gcetasks [no test files] === RUN Test_OpenstackCloud_GetApiIngressStatus === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled --- PASS: Test_OpenstackCloud_GetApiIngressStatus (0.02s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled (0.01s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstack 0.064s === RUN Test_Port_GetDependencies --- PASS: Test_Port_GetDependencies (0.00s) === RUN Test_NewPortTaskFromCloud === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found === RUN Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups --- PASS: Test_NewPortTaskFromCloud (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found (0.00s) --- PASS: Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups (0.00s) === RUN Test_Port_Find === RUN Test_Port_Find/nothing_found === RUN Test_Port_Find/port_found_no_tags === RUN Test_Port_Find/port_found_with_tags === RUN Test_Port_Find/multiple_ports_found === RUN Test_Port_Find/error_listing_ports --- PASS: Test_Port_Find (0.00s) --- PASS: Test_Port_Find/nothing_found (0.00s) --- PASS: Test_Port_Find/port_found_no_tags (0.00s) --- PASS: Test_Port_Find/port_found_with_tags (0.00s) --- PASS: Test_Port_Find/multiple_ports_found (0.00s) --- PASS: Test_Port_Find/error_listing_ports (0.00s) === RUN Test_Port_CheckChanges === RUN Test_Port_CheckChanges/actual_nil_all_required_fields_set === RUN Test_Port_CheckChanges/actual_nil_required_field_Name_nil === RUN Test_Port_CheckChanges/actual_nil_required_field_Network_nil === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set --- PASS: Test_Port_CheckChanges (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_all_required_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Name_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Network_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set (0.00s) === RUN Test_Port_RenderOpenstack === RUN Test_Port_RenderOpenstack/actual_not_nil === RUN Test_Port_RenderOpenstack/actual_nil_success === RUN Test_Port_RenderOpenstack/actual_nil_cloud_error --- PASS: Test_Port_RenderOpenstack (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_not_nil (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_success (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_cloud_error (0.00s) === RUN Test_Port_createOptsFromPortTask === RUN Test_Port_createOptsFromPortTask/all_fields_set === RUN Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups --- PASS: Test_Port_createOptsFromPortTask (0.00s) --- PASS: Test_Port_createOptsFromPortTask/all_fields_set (0.00s) --- PASS: Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstacktasks 0.055s ? k8s.io/kops/upup/pkg/fi/cloudup/spotinsttasks [no test files] === RUN TestWriteValue === RUN TestWriteValue/null === RUN TestWriteValue/empty_list === RUN TestWriteValue/string === RUN TestWriteValue/list === RUN TestWriteValue/list_of_objects === RUN TestWriteValue/object_block --- PASS: TestWriteValue (0.00s) --- PASS: TestWriteValue/null (0.00s) --- PASS: TestWriteValue/empty_list (0.00s) --- PASS: TestWriteValue/string (0.00s) --- PASS: TestWriteValue/list (0.00s) --- PASS: TestWriteValue/list_of_objects (0.00s) --- PASS: TestWriteValue/object_block (0.00s) === RUN TestWriteLiteral === RUN TestWriteLiteral/string === RUN TestWriteLiteral/traversal === RUN TestWriteLiteral/provider_alias === RUN TestWriteLiteral/file --- PASS: TestWriteLiteral (0.00s) --- PASS: TestWriteLiteral/string (0.00s) --- PASS: TestWriteLiteral/traversal (0.00s) --- PASS: TestWriteLiteral/provider_alias (0.00s) --- PASS: TestWriteLiteral/file (0.00s) === RUN TestWriteLiteralList === RUN TestWriteLiteralList/empty_list === RUN TestWriteLiteralList/one_literal === RUN TestWriteLiteralList/two_literals === RUN TestWriteLiteralList/one_traversal_literal,_one_string_literal --- PASS: TestWriteLiteralList (0.00s) --- PASS: TestWriteLiteralList/empty_list (0.00s) --- PASS: TestWriteLiteralList/one_literal (0.00s) --- PASS: TestWriteLiteralList/two_literals (0.00s) --- PASS: TestWriteLiteralList/one_traversal_literal,_one_string_literal (0.00s) === RUN TestWriteMap === RUN TestWriteMap/empty_map === RUN TestWriteMap/simple_map === RUN TestWriteMap/complex_keys === RUN TestWriteMap/literal_list --- PASS: TestWriteMap (0.00s) --- PASS: TestWriteMap/empty_map (0.00s) --- PASS: TestWriteMap/simple_map (0.00s) --- PASS: TestWriteMap/complex_keys (0.00s) --- PASS: TestWriteMap/literal_list (0.00s) === RUN TestWriteMapLiterals === RUN TestWriteMapLiterals/literal_values --- PASS: TestWriteMapLiterals (0.00s) --- PASS: TestWriteMapLiterals/literal_values (0.00s) === RUN TestWriteLocalsOutputs === RUN TestWriteLocalsOutputs/empty_map === RUN TestWriteLocalsOutputs/single_output === RUN TestWriteLocalsOutputs/list_output --- PASS: TestWriteLocalsOutputs (0.00s) --- PASS: TestWriteLocalsOutputs/empty_map (0.00s) --- PASS: TestWriteLocalsOutputs/single_output (0.00s) --- PASS: TestWriteLocalsOutputs/list_output (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraform 0.035s === RUN TestGetOutputs === RUN TestGetOutputs/empty_map === RUN TestGetOutputs/single_output === RUN TestGetOutputs/list_output === RUN TestGetOutputs/duplicate_names === RUN TestGetOutputs/duplicate_values --- PASS: TestGetOutputs (0.00s) --- PASS: TestGetOutputs/empty_map (0.00s) --- PASS: TestGetOutputs/single_output (0.00s) --- PASS: TestGetOutputs/list_output (0.00s) --- PASS: TestGetOutputs/duplicate_names (0.00s) --- PASS: TestGetOutputs/duplicate_values (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraformWriter 0.050s === RUN TestKeypairDeps --- PASS: TestKeypairDeps (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/fitasks 0.029s ? k8s.io/kops/upup/pkg/fi/loader [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/cloudinit [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/local [no test files] === RUN TestArchiveDependencies --- PASS: TestArchiveDependencies (0.00s) === RUN TestBindMountCommands I0512 07:42:16.174615 8700 bindmount.go:224] running mount command [mount --bind /containerized_mounter /containerized_mounter] I0512 07:42:16.174681 8700 bindmount.go:233] running mount command [mount -o remount,exec /containerized_mounter] I0512 07:42:16.174692 8700 bindmount.go:224] running mount command [mount --rbind /var/lib/kubelet/ /containerized_mounter/rootfs/var/lib/kubelet] I0512 07:42:16.174702 8700 bindmount.go:244] running mount command [mount --make-rshared /containerized_mounter/rootfs/var/lib/kubelet] I0512 07:42:16.174712 8700 bindmount.go:224] running mount command [mount --bind -o ro /proc /containerized_mounter/rootfs/proc] I0512 07:42:16.174724 8700 bindmount.go:224] running mount command [mount --bind -o ro /dev /containerized_mounter/rootfs/dev] I0512 07:42:16.174737 8700 bindmount.go:224] running mount command [mount --bind -o ro /etc/resolv.conf /containerized_mounter/rootfs/etc/resolv.conf] --- PASS: TestBindMountCommands (0.00s) === RUN TestBindMountDependencies --- PASS: TestBindMountDependencies (0.00s) === RUN TestFileDependencies === RUN TestFileDependencies/user === RUN TestFileDependencies/parentDir === RUN TestFileDependencies/afterFiles --- PASS: TestFileDependencies (0.00s) --- PASS: TestFileDependencies/user (0.00s) --- PASS: TestFileDependencies/parentDir (0.00s) --- PASS: TestFileDependencies/afterFiles (0.00s) === RUN TestIssueCertFileDependencies --- PASS: TestIssueCertFileDependencies (0.00s) === RUN TestLoadImageTask_Deps --- PASS: TestLoadImageTask_Deps (0.00s) === RUN TestServiceTask_Deps --- PASS: TestServiceTask_Deps (0.00s) === RUN TestServiceTask_UnknownTypes W0512 07:42:16.175250 8700 service.go:93] Unhandled type *nodetasks.FakeTask in Service::GetDependencies: &{} --- PASS: TestServiceTask_UnknownTypes (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/nodeup/nodetasks 0.057s ? k8s.io/kops/upup/pkg/fi/nodeup/nodetasks/dnstasks [no test files] ? k8s.io/kops/upup/pkg/fi/secrets [no test files] === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) === RUN TestStringSlicesEqualIgnoreOrder --- PASS: TestStringSlicesEqualIgnoreOrder (0.00s) === RUN TestHashString --- PASS: TestHashString (0.00s) === RUN TestSanitizeString --- PASS: TestSanitizeString (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/utils 0.053s ? k8s.io/kops/upup/pkg/kutil [no test files] ? k8s.io/kops/upup/tools/generators/fitask [no test files] ? k8s.io/kops/util/pkg/architectures [no test files] === RUN TestFindDistribution --- PASS: TestFindDistribution (0.00s) PASS ok k8s.io/kops/util/pkg/distributions 0.028s ? k8s.io/kops/util/pkg/env [no test files] === RUN TestWithTee --- PASS: TestWithTee (0.00s) PASS ok k8s.io/kops/util/pkg/exec 0.013s === RUN TestNewHasher === RUN TestNewHasher/sha256 === RUN TestNewHasher/sha1 === RUN TestNewHasher/md5 --- PASS: TestNewHasher (0.00s) --- PASS: TestNewHasher/sha256 (0.00s) --- PASS: TestNewHasher/sha1 (0.00s) --- PASS: TestNewHasher/md5 (0.00s) === RUN TestMemberFromString === RUN TestMemberFromString/sha256_1 === RUN TestMemberFromString/sha256_2 === RUN TestMemberFromString/sha256_3 === RUN TestMemberFromString/sha1_1 === RUN TestMemberFromString/sha1_2 === RUN TestMemberFromString/sha1_3 === RUN TestMemberFromString/md5_1 === RUN TestMemberFromString/md5_2 === RUN TestMemberFromString/md5_3 --- PASS: TestMemberFromString (0.00s) --- PASS: TestMemberFromString/sha256_1 (0.00s) --- PASS: TestMemberFromString/sha256_2 (0.00s) --- PASS: TestMemberFromString/sha256_3 (0.00s) --- PASS: TestMemberFromString/sha1_1 (0.00s) --- PASS: TestMemberFromString/sha1_2 (0.00s) --- PASS: TestMemberFromString/sha1_3 (0.00s) --- PASS: TestMemberFromString/md5_1 (0.00s) --- PASS: TestMemberFromString/md5_2 (0.00s) --- PASS: TestMemberFromString/md5_3 (0.00s) === RUN TestFromString === RUN TestFromString/sha256 === RUN TestFromString/sha1 === RUN TestFromString/md5 === RUN TestFromString/unknown --- PASS: TestFromString (0.00s) --- PASS: TestFromString/sha256 (0.00s) --- PASS: TestFromString/sha1 (0.00s) --- PASS: TestFromString/md5 (0.00s) --- PASS: TestFromString/unknown (0.00s) PASS ok k8s.io/kops/util/pkg/hashing 0.030s === RUN TestKeys --- PASS: TestKeys (0.00s) === RUN TestSortedKeys --- PASS: TestSortedKeys (0.00s) PASS ok k8s.io/kops/util/pkg/maps 0.025s ? k8s.io/kops/util/pkg/mirrors [no test files] === RUN TestGetProxyEnvVars W0512 07:42:11.764759 8398 proxy.go:34] EgressProxy set but no proxy host provided --- PASS: TestGetProxyEnvVars (0.00s) PASS ok k8s.io/kops/util/pkg/proxy 0.023s === RUN TestSet === RUN TestSet/simple_setting === RUN TestSet/setting_with_wildcard === RUN TestSet/creating_missing_objects === RUN TestSet/set_int === RUN TestSet/set_int32 === RUN TestSet/set_int64 === RUN TestSet/set_int_pointer === RUN TestSet/set_int32_pointer === RUN TestSet/set_int64_pointer === RUN TestSet/set_enum === RUN TestSet/set_enum_slice === RUN TestSet/append_enum_slice --- PASS: TestSet (0.00s) --- PASS: TestSet/simple_setting (0.00s) --- PASS: TestSet/setting_with_wildcard (0.00s) --- PASS: TestSet/creating_missing_objects (0.00s) --- PASS: TestSet/set_int (0.00s) --- PASS: TestSet/set_int32 (0.00s) --- PASS: TestSet/set_int64 (0.00s) --- PASS: TestSet/set_int_pointer (0.00s) --- PASS: TestSet/set_int32_pointer (0.00s) --- PASS: TestSet/set_int64_pointer (0.00s) --- PASS: TestSet/set_enum (0.00s) --- PASS: TestSet/set_enum_slice (0.00s) --- PASS: TestSet/append_enum_slice (0.00s) === RUN TestSetInvalidPath === RUN TestSetInvalidPath/setting_with_wildcard === RUN TestSetInvalidPath/creating_missing_objects === RUN TestSetInvalidPath/set_int === RUN TestSetInvalidPath/set_int32 === RUN TestSetInvalidPath/set_int64 --- PASS: TestSetInvalidPath (0.00s) --- PASS: TestSetInvalidPath/setting_with_wildcard (0.00s) --- PASS: TestSetInvalidPath/creating_missing_objects (0.00s) --- PASS: TestSetInvalidPath/set_int (0.00s) --- PASS: TestSetInvalidPath/set_int32 (0.00s) --- PASS: TestSetInvalidPath/set_int64 (0.00s) === RUN TestUnset === RUN TestUnset/simple_unsetting === RUN TestUnset/unsetting_with_wildcard === RUN TestUnset/uset_int === RUN TestUnset/unset_int32 === RUN TestUnset/unset_int64 === RUN TestUnset/unset_int_pointer === RUN TestUnset/unset_int32_pointer === RUN TestUnset/unset_int64_pointer === RUN TestUnset/unset_enum === RUN TestUnset/unset_enum_slice --- PASS: TestUnset (0.00s) --- PASS: TestUnset/simple_unsetting (0.00s) --- PASS: TestUnset/unsetting_with_wildcard (0.00s) --- PASS: TestUnset/uset_int (0.00s) --- PASS: TestUnset/unset_int32 (0.00s) --- PASS: TestUnset/unset_int64 (0.00s) --- PASS: TestUnset/unset_int_pointer (0.00s) --- PASS: TestUnset/unset_int32_pointer (0.00s) --- PASS: TestUnset/unset_int64_pointer (0.00s) --- PASS: TestUnset/unset_enum (0.00s) --- PASS: TestUnset/unset_enum_slice (0.00s) === RUN TestUnsetInvalidPath === RUN TestUnsetInvalidPath/usetting_with_wildcard === RUN TestUnsetInvalidPath/missing_objects --- PASS: TestUnsetInvalidPath (0.00s) --- PASS: TestUnsetInvalidPath/usetting_with_wildcard (0.00s) --- PASS: TestUnsetInvalidPath/missing_objects (0.00s) === RUN TestParseFieldPath === RUN TestParseFieldPath/test_Spec.Containers.Image === RUN TestParseFieldPath/test_Spec.Containers[0].Image === RUN TestParseFieldPath/test_Spec.Containers[*].Image --- PASS: TestParseFieldPath (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers.Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[0].Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[*].Image (0.00s) PASS ok k8s.io/kops/util/pkg/reflectutils 0.007s === RUN TestGetUniqueStrings --- PASS: TestGetUniqueStrings (0.00s) === RUN TestContains --- PASS: TestContains (0.00s) PASS ok k8s.io/kops/util/pkg/slice 0.007s ? k8s.io/kops/util/pkg/tables [no test files] === RUN TestSplitToSections --- PASS: TestSplitToSections (0.00s) PASS ok k8s.io/kops/util/pkg/text 0.010s ? k8s.io/kops/util/pkg/ui [no test files] === RUN TestAzureBlobPathBase === RUN TestAzureBlobPathBase/Test_case_0 === RUN TestAzureBlobPathBase/Test_case_1 === RUN TestAzureBlobPathBase/Test_case_2 --- PASS: TestAzureBlobPathBase (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_2 (0.00s) === RUN TestAzureBlobPathPath === RUN TestAzureBlobPathPath/Test_case_0 === RUN TestAzureBlobPathPath/Test_case_1 === RUN TestAzureBlobPathPath/Test_case_2 --- PASS: TestAzureBlobPathPath (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_2 (0.00s) === RUN TestAzureBlobPathJoin --- PASS: TestAzureBlobPathJoin (0.00s) === RUN TestBuildVaultPath context_test.go:27: No vault dev token set --- SKIP: TestBuildVaultPath (0.00s) === RUN TestCreateFile --- PASS: TestCreateFile (0.00s) === RUN TestWriteTo --- PASS: TestWriteTo (0.00s) === RUN TestMemFsCreateFile --- PASS: TestMemFsCreateFile (0.00s) === RUN TestMemFsReadDir --- PASS: TestMemFsReadDir (0.00s) === RUN TestMemFsReadTree --- PASS: TestMemFsReadTree (0.00s) === RUN Test_VFSPath --- PASS: Test_VFSPath (0.00s) === RUN Test_S3Path_Parse --- PASS: Test_S3Path_Parse (0.00s) === RUN Test_newVaultPath vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_newVaultPath (0.00s) === RUN Test_newVaultPathHostOnly --- PASS: Test_newVaultPathHostOnly (0.00s) === RUN Test_WriteFileReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileReadFile (0.00s) === RUN Test_WriteFileDeleteFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileDeleteFile (0.00s) === RUN Test_CreateFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_CreateFile (0.00s) === RUN Test_ReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadFile (0.00s) === RUN Test_Join vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_Join (0.00s) === RUN Test_VaultReadDirList vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadDirList (0.00s) === RUN Test_ReadDir vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadDir (0.00s) === RUN Test_IsDirectory vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_IsDirectory (0.00s) === RUN Test_VaultReadTree vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadTree (0.00s) PASS ok k8s.io/kops/util/pkg/vfs 0.023s === RUN TestGSRenderTerraform === RUN TestGSRenderTerraform/gs://foo/bar --- PASS: TestGSRenderTerraform (0.00s) --- PASS: TestGSRenderTerraform/gs://foo/bar (0.00s) === RUN TestS3RenderTerraform === RUN TestS3RenderTerraform/s3://foo/bar I0512 07:42:16.511972 8807 s3context.go:89] Found S3_ENDPOINT="foo.s3.amazonaws.com", using as non-AWS S3 backend W0512 07:42:16.512012 8807 s3context.go:237] Unable to read bucket encryption policy for "foo" in region "us-east-1": will encrypt using AES256 --- PASS: TestS3RenderTerraform (0.00s) --- PASS: TestS3RenderTerraform/s3://foo/bar (0.00s) PASS ok k8s.io/kops/util/pkg/vfs/tests 0.045s >>> kops: Entering fakeroot... >>> kops*: Running postcheck for kops >>> kops*: Preparing package kops... >>> kops*: Stripping binaries >>> kops*: Scanning shared objects >>> kops*: Tracing dependencies... kubernetes so:libc.musl-x86_64.so.1 >>> kops*: Package size: 156.6 MB >>> kops*: Compressing data... >>> kops*: Create checksum... >>> kops*: Create kops-1.23.0-r2.apk >>> kops: Build complete at Thu, 12 May 2022 07:43:17 +0000 elapsed time 0h 3m 16s >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Uninstalling dependencies... (1/3) Purging .makedepends-kops (20220512.074002) (2/3) Purging kubernetes (1.24.0-r2) (3/3) Purging go (1.18.2-r0) Executing busybox-1.35.0-r13.trigger OK: 332 MiB in 98 packages >>> kops: Updating the testing/x86_64 repository index... >>> kops: Signing the index...