>>> kops: Building testing/kops 1.24.0-r1 (using abuild 3.9.0-r3) started Wed, 03 Aug 2022 22:14:51 +0000 >>> kops: Checking sanity of /home/buildozer/aports/testing/kops/APKBUILD... >>> kops: Analyzing dependencies... >>> kops: Installing for build: build-base kubernetes go (1/2) Installing kubernetes (1.24.3-r2) (2/2) Installing .makedepends-kops (20220803.221453) OK: 715 MiB in 106 packages >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.24.0.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 63 29.3M 63 18.4M 0 0 46.2M 0 --:--:-- --:--:-- --:--:-- 46.2M 100 29.3M 100 29.3M 0 0 57.8M 0 --:--:-- --:--:-- --:--:-- 57.7M >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.24.0.tar.gz >>> kops: Checking sha512sums... kops-1.24.0.tar.gz: OK >>> kops: Unpacking /var/cache/distfiles/edge/kops-1.24.0.tar.gz... go: downloading k8s.io/apimachinery v0.24.0 go: downloading google.golang.org/api v0.86.0 go: downloading k8s.io/klog/v2 v2.70.0 go: downloading github.com/aws/aws-sdk-go v1.44.45 go: downloading github.com/prometheus/client_golang v1.12.2 go: downloading k8s.io/api v0.24.0 go: downloading github.com/spf13/pflag v1.0.5 go: downloading k8s.io/client-go v0.24.0 go: downloading github.com/blang/semver/v4 v4.0.0 go: downloading github.com/cert-manager/cert-manager v1.8.2 go: downloading k8s.io/component-base v0.24.0 go: downloading github.com/spf13/cobra v1.5.0 go: downloading go.uber.org/multierr v1.8.0 go: downloading k8s.io/cli-runtime v0.24.0 go: downloading github.com/spf13/viper v1.12.0 go: downloading github.com/digitalocean/godo v1.81.0 go: downloading k8s.io/klog v1.0.0 go: downloading golang.org/x/oauth2 v0.0.0-20220628200809-02e64fa58f26 go: downloading github.com/google/go-cmp v0.5.8 go: downloading github.com/google/uuid v1.3.0 go: downloading github.com/gophercloud/gophercloud v0.25.0 go: downloading golang.org/x/sys v0.0.0-20220627191245-f75cf1eec38b go: downloading sigs.k8s.io/controller-runtime v0.12.2 go: downloading sigs.k8s.io/yaml v1.3.0 go: downloading cloud.google.com/go/compute v1.7.0 go: downloading gopkg.in/gcfg.v1 v1.2.3 go: downloading k8s.io/legacy-cloud-providers v0.24.0 go: downloading golang.org/x/net v0.0.0-20220624214902-1bab6f366d9e go: downloading github.com/go-logr/logr v1.2.3 go: downloading github.com/stretchr/testify v1.8.0 go: downloading gopkg.in/inf.v0 v0.9.1 go: downloading github.com/google/go-containerregistry v0.10.0 go: downloading k8s.io/kubectl v0.24.0 go: downloading github.com/hetznercloud/hcloud-go v1.35.0 go: downloading github.com/pelletier/go-toml v1.9.5 go: downloading k8s.io/kubelet v0.24.0 go: downloading k8s.io/mount-utils v0.24.0 go: downloading k8s.io/utils v0.0.0-20220210201930-3a6ce19ff2f9 go: downloading golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d go: downloading github.com/MakeNowJust/heredoc/v2 v2.0.1 go: downloading github.com/MakeNowJust/heredoc v0.0.0-20170808103936-bb23615498cd go: downloading github.com/sergi/go-diff v1.2.0 go: downloading github.com/aws/amazon-ec2-instance-selector/v2 v2.3.1 go: downloading github.com/Azure/azure-sdk-for-go v66.0.0+incompatible go: downloading github.com/Azure/go-autorest/autorest/to v0.4.0 go: downloading helm.sh/helm/v3 v3.9.0 go: downloading gopkg.in/square/go-jose.v2 v2.6.0 go: downloading github.com/Azure/go-autorest/autorest v0.11.27 go: downloading github.com/Azure/go-autorest v14.2.0+incompatible go: downloading golang.org/x/sync v0.0.0-20220601150217-0de741cfad7f go: downloading github.com/Azure/go-autorest/autorest/azure/auth v0.5.11 go: downloading go.uber.org/atomic v1.9.0 go: downloading github.com/inconshreveable/mousetrap v1.0.0 go: downloading github.com/fsnotify/fsnotify v1.5.4 go: downloading github.com/mitchellh/mapstructure v1.5.0 go: downloading github.com/spf13/afero v1.8.2 go: downloading github.com/spf13/cast v1.5.0 go: downloading github.com/spf13/jwalterweatherman v1.1.0 go: downloading github.com/prometheus/client_model v0.2.0 go: downloading github.com/prometheus/common v0.32.1 go: downloading github.com/apparentlymart/go-cidr v1.1.0 go: downloading github.com/google/go-querystring v1.1.0 go: downloading github.com/evanphx/json-patch v5.6.0+incompatible go: downloading github.com/gogo/protobuf v1.3.2 go: downloading github.com/google/gofuzz v1.2.0 go: downloading github.com/Azure/azure-pipeline-go v0.2.3 go: downloading github.com/Azure/azure-storage-blob-go v0.15.0 go: downloading github.com/go-ini/ini v1.66.6 go: downloading github.com/hashicorp/vault/api v1.7.2 go: downloading github.com/pkg/sftp v1.13.5 go: downloading github.com/Masterminds/sprig/v3 v3.2.2 go: downloading github.com/jacksontj/memberlistmesh v0.0.0-20190905163944-93462b9d2bb7 go: downloading github.com/weaveworks/mesh v0.0.0-20191105120815-58dbcc3e8e63 go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.2.1 go: downloading github.com/davecgh/go-spew v1.1.1 go: downloading github.com/google/gnostic v0.5.7-v3refs go: downloading github.com/imdario/mergo v0.3.12 go: downloading gopkg.in/yaml.v2 v2.4.0 go: downloading golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 go: downloading google.golang.org/appengine v1.6.7 go: downloading gopkg.in/warnings.v0 v0.1.2 go: downloading github.com/golang/protobuf v1.5.2 go: downloading google.golang.org/protobuf v1.28.0 go: downloading github.com/pmezard/go-difflib v1.0.0 go: downloading gopkg.in/yaml.v3 v3.0.1 go: downloading golang.org/x/time v0.0.0-20220210224613-90d013bbcef8 go: downloading github.com/GoogleCloudPlatform/k8s-cloud-provider v1.16.1-0.20210702024009-ea6160c1d0e3 go: downloading k8s.io/cloud-provider v0.24.0 go: downloading github.com/spotinst/spotinst-sdk-go v1.121.0 go: downloading github.com/docker/cli v20.10.16+incompatible go: downloading github.com/mitchellh/go-homedir v1.1.0 go: downloading github.com/containerd/stargz-snapshotter/estargz v0.11.4 go: downloading github.com/google/go-tpm-tools v0.3.8 go: downloading github.com/google/go-tpm v0.3.3 go: downloading github.com/chai2010/gettext-go v0.0.0-20160711120539-c6fed771bfd5 go: downloading github.com/mitchellh/go-wordwrap v1.0.0 go: downloading github.com/russross/blackfriday v1.6.0 go: downloading sigs.k8s.io/json v0.0.0-20211208200746-9f7c6b3444d2 go: downloading k8s.io/kube-openapi v0.0.0-20220328201542-3ee0da9b0b42 go: downloading github.com/googleapis/gax-go/v2 v2.4.0 go: downloading google.golang.org/grpc v1.47.0 go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.2 go: downloading k8s.io/csi-translation-lib v0.24.0 go: downloading github.com/hashicorp/hcl/v2 v2.13.0 go: downloading github.com/zclconf/go-cty v1.10.0 go: downloading k8s.io/gengo v0.0.0-20220613173612-397b4ae3bce7 go: downloading github.com/Azure/go-autorest/autorest/adal v0.9.18 go: downloading github.com/Azure/go-autorest/autorest/azure/cli v0.4.5 go: downloading github.com/dimchansky/utfbom v1.1.1 go: downloading github.com/Azure/go-autorest/logger v0.2.1 go: downloading github.com/pkg/errors v0.9.1 go: downloading golang.org/x/text v0.3.7 go: downloading github.com/subosito/gotenv v1.3.0 go: downloading github.com/hashicorp/hcl v1.0.0 go: downloading gopkg.in/ini.v1 v1.66.6 go: downloading github.com/magiconair/properties v1.8.6 go: downloading github.com/pelletier/go-toml/v2 v2.0.1 go: downloading github.com/beorn7/perks v1.0.1 go: downloading github.com/cespare/xxhash/v2 v2.1.2 go: downloading github.com/prometheus/procfs v0.7.3 go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de go: downloading sigs.k8s.io/kustomize/api v0.11.4 go: downloading sigs.k8s.io/kustomize/kyaml v0.13.6 go: downloading github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 go: downloading github.com/peterbourgon/diskv v2.0.1+incompatible go: downloading k8s.io/apiextensions-apiserver v0.24.0 go: downloading sigs.k8s.io/gateway-api v0.4.1 go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da go: downloading gomodules.xyz/jsonpatch/v2 v2.2.0 go: downloading github.com/mattn/go-ieproxy v0.0.1 go: downloading github.com/Masterminds/goutils v1.1.1 go: downloading github.com/Masterminds/semver/v3 v3.1.1 go: downloading github.com/huandu/xstrings v1.3.2 go: downloading github.com/mitchellh/copystructure v1.2.0 go: downloading github.com/shopspring/decimal v1.2.0 go: downloading github.com/cenkalti/backoff/v3 v3.0.0 go: downloading github.com/hashicorp/errwrap v1.1.0 go: downloading github.com/hashicorp/go-cleanhttp v0.5.2 go: downloading github.com/hashicorp/go-multierror v1.1.1 go: downloading github.com/hashicorp/go-retryablehttp v0.6.6 go: downloading github.com/hashicorp/go-rootcerts v1.0.2 go: downloading github.com/hashicorp/go-secure-stdlib/parseutil v0.1.6 go: downloading github.com/hashicorp/vault/sdk v0.5.1 go: downloading github.com/hashicorp/go-hclog v1.2.0 go: downloading github.com/kr/fs v0.1.0 go: downloading github.com/json-iterator/go v1.1.12 go: downloading github.com/hashicorp/go-sockaddr v1.0.2 go: downloading github.com/hashicorp/memberlist v0.3.1 go: downloading github.com/oklog/ulid v1.3.1 go: downloading github.com/docker/distribution v2.8.1+incompatible go: downloading github.com/klauspost/compress v1.15.4 go: downloading github.com/opencontainers/go-digest v1.0.0 go: downloading github.com/vbatts/tar-split v0.11.2 go: downloading github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6 go: downloading github.com/Azure/go-autorest/tracing v0.6.0 go: downloading go.opencensus.io v0.23.0 go: downloading golang.org/x/xerrors v0.0.0-20220609144429-65e65417b02f go: downloading github.com/patrickmn/go-cache v2.1.0+incompatible go: downloading github.com/stretchr/objx v0.4.0 go: downloading github.com/onsi/ginkgo v1.16.5 go: downloading github.com/onsi/gomega v1.18.1 go: downloading gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c go: downloading go.uber.org/goleak v1.1.12 go: downloading github.com/frankban/quicktest v1.14.3 go: downloading github.com/go-logr/zapr v1.2.0 go: downloading go.uber.org/zap v1.19.1 go: downloading github.com/go-test/deep v1.0.3 go: downloading github.com/Azure/go-autorest/autorest/mocks v0.4.2 go: downloading google.golang.org/genproto v0.0.0-20220624142145-8cd45d7dbd1f go: downloading github.com/russross/blackfriday/v2 v2.1.0 go: downloading github.com/Azure/go-autorest/autorest/date v0.3.0 go: downloading github.com/golang-jwt/jwt/v4 v4.2.0 go: downloading golang.org/x/tools v0.1.8 go: downloading github.com/jmespath/go-jmespath v0.4.0 go: downloading github.com/google/btree v1.0.1 go: downloading github.com/agext/levenshtein v1.2.1 go: downloading github.com/apparentlymart/go-textseg/v13 v13.0.0 go: downloading github.com/kylelemons/godebug v1.1.0 go: downloading github.com/kr/pretty v0.3.0 go: downloading github.com/docker/docker v20.10.16+incompatible go: downloading gotest.tools/v3 v3.0.3 go: downloading github.com/sirupsen/logrus v1.8.1 go: downloading github.com/mitchellh/reflectwalk v1.0.2 go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 go: downloading github.com/go-openapi/swag v0.19.14 go: downloading github.com/hashicorp/go-secure-stdlib/strutil v0.1.2 go: downloading github.com/google/go-attestation v0.4.4-0.20220404204839-8820d49b18d9 go: downloading github.com/mitchellh/go-testing-interface v1.0.0 go: downloading github.com/fatih/color v1.13.0 go: downloading github.com/opencontainers/image-spec v1.0.3-0.20220114050600-8b9d41f48198 go: downloading github.com/mattn/go-colorable v0.1.12 go: downloading github.com/mattn/go-isatty v0.0.14 go: downloading github.com/armon/go-metrics v0.3.10 go: downloading github.com/hashicorp/go-msgpack v0.5.3 go: downloading github.com/miekg/dns v1.1.48 go: downloading github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 go: downloading github.com/modern-go/reflect2 v1.0.2 go: downloading github.com/googleapis/enterprise-certificate-proxy v0.1.0 go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd go: downloading github.com/creack/pty v1.1.11 go: downloading github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d go: downloading github.com/containerd/containerd v1.6.3 go: downloading oras.land/oras-go v1.1.1 go: downloading github.com/distribution/distribution/v3 v3.0.0-20211118083504-a29a3c99a684 go: downloading github.com/phayes/freeport v0.0.0-20180830031419-95f893ade6f2 go: downloading github.com/kr/text v0.2.0 go: downloading github.com/rogpeppe/go-internal v1.6.2 go: downloading github.com/docker/docker-credential-helpers v0.6.4 go: downloading github.com/emicklei/go-restful v2.9.5+incompatible go: downloading github.com/go-openapi/jsonreference v0.19.5 go: downloading github.com/golang/snappy v0.0.4 go: downloading github.com/pierrec/lz4 v2.5.2+incompatible go: downloading github.com/hashicorp/go-plugin v1.4.3 go: downloading github.com/hashicorp/go-secure-stdlib/mlock v0.1.1 go: downloading github.com/hashicorp/go-uuid v1.0.2 go: downloading github.com/hashicorp/go-version v1.2.1 go: downloading github.com/hashicorp/golang-lru v0.5.4 go: downloading github.com/armon/go-radix v1.0.0 go: downloading github.com/mailru/easyjson v0.7.6 go: downloading github.com/moby/spdystream v0.2.0 go: downloading github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 go: downloading github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153 go: downloading github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 go: downloading github.com/jmespath/go-jmespath/internal/testify v1.5.1 go: downloading github.com/ryanuber/go-glob v1.0.0 go: downloading github.com/hashicorp/go-immutable-radix v1.3.1 go: downloading github.com/pascaldekloe/goe v0.1.0 go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: downloading github.com/go-errors/errors v1.0.1 go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 go: downloading github.com/xlab/treeprint v0.0.0-20181112141820-a009c3971eca go: downloading github.com/nxadm/tail v1.4.8 go: downloading github.com/google/go-tspi v0.2.1-0.20190423175329-115dea689aad go: downloading github.com/PuerkitoBio/purell v1.1.1 go: downloading github.com/go-openapi/jsonpointer v0.19.5 go: downloading github.com/fatih/structs v1.1.0 go: downloading github.com/hashicorp/yamux v0.0.0-20180604194846-3520598351bb go: downloading github.com/oklog/run v1.0.0 go: downloading github.com/jhump/protoreflect v1.6.1 go: downloading github.com/josharian/intern v1.0.0 go: downloading github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d go: downloading github.com/bshuster-repo/logrus-logstash-hook v1.0.0 go: downloading github.com/bugsnag/bugsnag-go v0.0.0-20141110184014-b1d153021fcd go: downloading github.com/docker/go-metrics v0.0.1 go: downloading github.com/docker/libtrust v0.0.0-20150114040149-fa567046d9b1 go: downloading github.com/gorilla/handlers v1.5.1 go: downloading github.com/yvasiyarov/gorelic v0.0.0-20141212073537-a9bba5b9ab50 go: downloading gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 go: downloading github.com/google/certificate-transparency-go v1.1.2 go: downloading github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 go: downloading github.com/gorilla/mux v1.8.0 go: downloading github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c go: downloading github.com/gomodule/redigo v1.8.2 go: downloading github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0 go: downloading go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 go: downloading github.com/felixge/httpsnoop v1.0.1 go: downloading github.com/yvasiyarov/go-metrics v0.0.0-20140926110328-57bccd1ccd43 go: downloading github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f go: downloading github.com/moby/locker v1.0.1 go: downloading golang.org/x/mod v0.6.0-dev.0.20220106191415-9b9b3d81d5e3 go: downloading github.com/bugsnag/osext v0.0.0-20130617224835-0dd3f918b21b go: downloading github.com/moby/sys/mountinfo v0.5.0 go: downloading github.com/Microsoft/hcsshim v0.9.2 go: downloading github.com/docker/go-connections v0.4.0 go: downloading github.com/morikuni/aec v1.0.0 go: downloading github.com/docker/go-units v0.4.0 go: downloading github.com/Microsoft/go-winio v0.5.2 go: downloading github.com/containerd/cgroups v1.0.3 go: downloading github.com/Azure/go-autorest/autorest/validation v0.3.1 /bin/sh: tools/get_version.sh: not found mkdir -p /home/buildozer/aports/testing/kops/src/kops-1.24.0/.build/dist/linux/amd64 GOOS=linux GOARCH=amd64 go build "-trimpath" -o /home/buildozer/aports/testing/kops/src/kops-1.24.0/.build/dist/linux/amd64/kops -ldflags=all=" -X k8s.io/kops.Version= -X k8s.io/kops.GitVersion=v20220715-1545-g5d7cc771bc" k8s.io/kops/cmd/kops go test -v ./... ? k8s.io/kops [no test files] ? k8s.io/kops/channels/cmd/channels [no test files] ? k8s.io/kops/channels/pkg/api [no test files] === RUN Test_Filtering --- PASS: Test_Filtering (0.00s) === RUN Test_Replacement --- PASS: Test_Replacement (0.00s) === RUN Test_GetRequiredUpdates --- PASS: Test_GetRequiredUpdates (0.00s) === RUN Test_NeedsRollingUpdate I0803 22:17:00.442381 14534 addon.go:229] addon test wants to update all nodes I0803 22:17:00.444310 14534 addon.go:229] addon test wants to update worker nodes I0803 22:17:00.444629 14534 addon.go:229] addon test wants to update control-plane nodes --- PASS: Test_NeedsRollingUpdate (0.00s) === RUN Test_InstallPKI I0803 22:17:00.445027 14534 addon.go:262] installing PKI for "test" I0803 22:17:00.779729 14534 addon.go:262] installing PKI for "test" --- PASS: Test_InstallPKI (0.77s) === RUN Test_IsPKIInstalled --- PASS: Test_IsPKIInstalled (0.00s) PASS ok k8s.io/kops/channels/pkg/channels 0.815s === RUN TestGetUpdates --- PASS: TestGetUpdates (0.00s) PASS ok k8s.io/kops/channels/pkg/cmd 0.048s ? k8s.io/kops/cloudmock/aws/mockautoscaling [no test files] ? k8s.io/kops/cloudmock/aws/mockec2 [no test files] ? k8s.io/kops/cloudmock/aws/mockelb [no test files] ? k8s.io/kops/cloudmock/aws/mockelbv2 [no test files] ? k8s.io/kops/cloudmock/aws/mockeventbridge [no test files] ? k8s.io/kops/cloudmock/aws/mockiam [no test files] ? k8s.io/kops/cloudmock/aws/mockroute53 [no test files] ? k8s.io/kops/cloudmock/aws/mocksqs [no test files] ? k8s.io/kops/cloudmock/gce [no test files] ? k8s.io/kops/cloudmock/gce/gcphttp [no test files] ? k8s.io/kops/cloudmock/gce/mockcloudresourcemanager [no test files] ? k8s.io/kops/cloudmock/gce/mockcompute [no test files] ? k8s.io/kops/cloudmock/gce/mockdns [no test files] ? k8s.io/kops/cloudmock/gce/mockiam [no test files] ? k8s.io/kops/cloudmock/gce/mockstorage [no test files] ? k8s.io/kops/cloudmock/openstack [no test files] ? k8s.io/kops/cloudmock/openstack/mockblockstorage [no test files] ? k8s.io/kops/cloudmock/openstack/mockcompute [no test files] ? k8s.io/kops/cloudmock/openstack/mockdns [no test files] ? k8s.io/kops/cloudmock/openstack/mockimage [no test files] ? k8s.io/kops/cloudmock/openstack/mockloadbalancer [no test files] ? k8s.io/kops/cloudmock/openstack/mocknetworking [no test files] === RUN TestCreateClusterMinimal I0803 22:17:29.434992 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:29.435263 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:29.435330 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:29.435381 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:29.435433 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:29.435566 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:29.435592 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:29.435618 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:29.435644 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:29.435718 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:29.435753 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:29.509684 20401 new_cluster.go:1168] Cloud Provider ID = aws I0803 22:17:29.509800 20401 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0803 22:17:29.510256 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:29.522848 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:29.523053 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:29.523089 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:29.523121 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:29.523153 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:29.523182 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:29.523198 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:29.523221 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:29.523238 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:29.523299 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:29.523324 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:29.550397 20401 new_cluster.go:1168] Cloud Provider ID = aws I0803 22:17:29.550481 20401 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0803 22:17:29.550760 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:29.557191 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:29.557368 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:29.557405 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:29.557431 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:29.557461 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:29.557486 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:29.557503 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:29.557526 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:29.557544 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:29.557601 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:29.557626 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:29.635173 20401 new_cluster.go:1168] Cloud Provider ID = aws I0803 22:17:29.635268 20401 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0803 22:17:29.635672 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:29.642348 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:29.642498 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:29.642533 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:29.642570 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:29.642606 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:29.642635 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:29.642654 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:29.642680 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:29.642700 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:29.642768 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:29.642807 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:29.686831 20401 new_cluster.go:1168] Cloud Provider ID = aws I0803 22:17:29.686881 20401 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0803 22:17:29.687076 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:29.692383 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:29.692498 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:29.692524 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:29.692548 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:29.692576 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:29.692599 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:29.692617 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:29.692638 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:29.692653 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:29.692696 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:29.692719 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:29.719141 20401 new_cluster.go:1168] Cloud Provider ID = aws I0803 22:17:29.719189 20401 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0803 22:17:29.719391 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:29.724674 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:29.724787 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:29.724811 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:29.724834 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:29.724860 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:29.724882 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:29.724899 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:29.724919 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:29.724932 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:29.724972 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:29.724993 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:29.784500 20401 new_cluster.go:1168] Cloud Provider ID = aws I0803 22:17:29.784584 20401 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0803 22:17:29.784865 20401 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterMinimal (0.36s) === RUN TestCreateClusterOverride I0803 22:17:29.790830 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:29.791018 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:29.791051 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:29.791081 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:29.791113 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:29.791167 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:29.791186 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:29.791208 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:29.791225 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:29.791308 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:29.791335 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:29.812653 20401 new_cluster.go:1168] Cloud Provider ID = aws I0803 22:17:29.813034 20401 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0803 22:17:29.813283 20401 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterOverride (0.03s) === RUN TestCreateClusterComplex I0803 22:17:29.818794 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:29.818992 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:29.819028 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:29.819059 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:29.819091 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:29.819157 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:29.819187 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:29.819212 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:29.819228 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:29.819279 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:29.819305 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:29.842395 20401 new_cluster.go:1168] Cloud Provider ID = aws I0803 22:17:29.842448 20401 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0803 22:17:29.842635 20401 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterComplex (0.03s) === RUN TestCreateClusterHA I0803 22:17:29.847805 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:29.848052 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:29.848080 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:29.848105 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:29.848132 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:29.848175 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:29.848194 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:29.848215 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:29.848231 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:29.848276 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:29.848300 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:29.896110 20401 new_cluster.go:1168] Cloud Provider ID = aws I0803 22:17:29.896193 20401 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0803 22:17:29.896207 20401 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0803 22:17:29.896217 20401 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0803 22:17:29.896535 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:29.905739 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:29.906002 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:29.906037 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:29.906069 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:29.906102 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:29.906129 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:29.906146 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:29.906168 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:29.906184 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:29.906241 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:29.906268 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:29.945567 20401 new_cluster.go:1168] Cloud Provider ID = aws I0803 22:17:29.945642 20401 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0803 22:17:29.945658 20401 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0803 22:17:29.945669 20401 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0803 22:17:29.945966 20401 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHA (0.11s) === RUN TestCreateClusterHAGCE I0803 22:17:29.955034 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:29.955175 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:29.955205 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:29.955232 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:29.955262 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:29.955314 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:29.955332 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:29.955355 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:29.955373 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:29.955430 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:29.955469 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:29.995888 20401 new_cluster.go:1168] Cloud Provider ID = gce I0803 22:17:29.995966 20401 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterHAGCE (0.06s) === RUN TestCreateClusterGCE I0803 22:17:30.013335 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:30.013647 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:30.013711 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:30.013778 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:30.013841 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:30.013918 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:30.013951 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:30.013987 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:30.014013 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:30.014090 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:30.014129 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:30.054360 20401 new_cluster.go:425] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0803 22:17:30.054438 20401 new_cluster.go:1168] Cloud Provider ID = gce I0803 22:17:30.054486 20401 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterGCE (0.05s) === RUN TestCreateClusterHASharedZone I0803 22:17:30.062135 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:30.062333 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:30.062367 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:30.062400 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:30.062446 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:30.062515 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:30.062537 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:30.062561 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:30.062579 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:30.062635 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:30.062663 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0803 22:17:30.189955 20401 new_cluster.go:781] Running with masters in the same AZs; redundancy will be reduced I0803 22:17:30.190012 20401 new_cluster.go:1168] Cloud Provider ID = aws I0803 22:17:30.190080 20401 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0803 22:17:30.190391 20401 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZone (0.14s) === RUN TestCreateClusterHASharedZones I0803 22:17:30.198223 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:30.198369 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:30.198401 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:30.198433 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:30.198465 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:30.198515 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:30.198533 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:30.198557 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:30.198575 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:30.198632 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:30.198657 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0803 22:17:30.237884 20401 new_cluster.go:781] Running with masters in the same AZs; redundancy will be reduced I0803 22:17:30.237929 20401 new_cluster.go:1168] Cloud Provider ID = aws I0803 22:17:30.237980 20401 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0803 22:17:30.237994 20401 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0803 22:17:30.238247 20401 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZones (0.05s) === RUN TestCreateClusterPrivate I0803 22:17:30.250079 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:30.250225 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:30.250257 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:30.250285 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:30.250316 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:30.250364 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:30.250384 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:30.250405 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:30.250423 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:30.250478 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:30.250525 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:30.304540 20401 new_cluster.go:1168] Cloud Provider ID = aws I0803 22:17:30.304663 20401 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0803 22:17:30.304690 20401 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0803 22:17:30.305140 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:30.305193 20401 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterPrivate (0.07s) === RUN TestCreateClusterPrivateGCE I0803 22:17:30.315154 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:30.315354 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:30.315391 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:30.315433 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:30.315508 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:30.315571 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:30.315596 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:30.315623 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:30.315643 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:30.315707 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:30.315743 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:30.357632 20401 new_cluster.go:425] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0803 22:17:30.357681 20401 new_cluster.go:1168] Cloud Provider ID = gce I0803 22:17:30.357718 20401 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterPrivateGCE (0.05s) === RUN TestCreateClusterWithNGWSpecified I0803 22:17:30.364614 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:30.364858 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:30.364881 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:30.364903 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:30.364928 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:30.364973 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:30.364990 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:30.365016 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:30.365031 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:30.365070 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:30.365092 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:30.407960 20401 new_cluster.go:1168] Cloud Provider ID = aws I0803 22:17:30.408041 20401 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0803 22:17:30.408057 20401 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0803 22:17:30.408436 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:30.408471 20401 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithNGWSpecified (0.05s) === RUN TestCreateClusterWithINGWSpecified I0803 22:17:30.415141 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:30.415354 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:30.415388 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:30.415419 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:30.415463 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:30.415523 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:30.415541 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:30.415564 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:30.415583 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:30.415643 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:30.415670 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:30.434478 20401 new_cluster.go:1168] Cloud Provider ID = aws I0803 22:17:30.434531 20401 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0803 22:17:30.434544 20401 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0803 22:17:30.434762 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:30.434788 20401 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithINGWSpecified (0.03s) === RUN TestCreateClusterSharedVPC I0803 22:17:30.441087 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:30.441226 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:30.441253 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:30.441278 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:30.441308 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:30.441350 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:30.441368 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:30.441388 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:30.441401 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:30.441447 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:30.441470 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:30.498111 20401 new_cluster.go:1168] Cloud Provider ID = aws I0803 22:17:30.498153 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0803 22:17:30.498201 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0803 22:17:30.498237 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0803 22:17:30.498253 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0803 22:17:30.498292 20401 subnets.go:185] Assigned CIDR 10.2.0.0/15 to subnet us-test-1a I0803 22:17:30.498513 20401 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedVPC (0.06s) === RUN TestCreateClusterSharedSubnets I0803 22:17:30.505624 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:30.505919 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:30.505961 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:30.506005 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:30.506053 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:30.506130 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:30.506162 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:30.506198 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:30.506224 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:30.506340 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:30.506383 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:30.591428 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0803 22:17:30.591457 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0803 22:17:30.591509 20401 new_cluster.go:1168] Cloud Provider ID = aws I0803 22:17:30.591546 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0803 22:17:30.591566 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0803 22:17:30.591588 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0803 22:17:30.591602 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0803 22:17:30.591806 20401 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnets (0.09s) === RUN TestCreateClusterSharedSubnetsVpcLookup I0803 22:17:30.597525 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:30.597650 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:30.597679 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:30.597701 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:30.597725 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:30.597767 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:30.597783 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:30.597806 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:30.597820 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:30.597860 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:30.597882 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:30.628523 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:17:30.628577 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0803 22:17:30.628600 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0803 22:17:30.628674 20401 new_cluster.go:1168] Cloud Provider ID = aws I0803 22:17:30.628724 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0803 22:17:30.628744 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0803 22:17:30.628781 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0803 22:17:30.628800 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0803 22:17:30.629133 20401 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnetsVpcLookup (0.04s) === RUN TestCreateClusterPrivateSharedSubnets I0803 22:17:30.634850 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:30.635001 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:30.635036 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:30.635062 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:30.635095 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:30.635151 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:30.635169 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:30.635193 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:30.635214 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:30.635288 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:30.635315 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:30.677449 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0803 22:17:30.677482 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0803 22:17:30.677540 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0803 22:17:30.677562 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0803 22:17:30.677593 20401 new_cluster.go:1168] Cloud Provider ID = aws I0803 22:17:30.677631 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0803 22:17:30.677650 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0803 22:17:30.677674 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0803 22:17:30.677691 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0803 22:17:30.677953 20401 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterPrivateSharedSubnets (0.05s) === RUN TestCreateClusterIPv6 I0803 22:17:30.683688 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:30.683817 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:30.683845 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:30.683870 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:30.683899 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:30.683947 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:30.683965 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:30.683986 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:30.684003 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:30.684056 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:30.684080 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:30.742219 20401 new_cluster.go:1168] Cloud Provider ID = aws I0803 22:17:30.742310 20401 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0803 22:17:30.742830 20401 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterIPv6 (0.07s) === RUN TestCreateClusterKarpenter I0803 22:17:30.753936 20401 featureflag.go:162] FeatureFlag "Karpenter"=true I0803 22:17:30.753960 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:30.754152 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:30.754198 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:30.754243 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:30.754295 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:30.754378 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:30.754414 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:30.754455 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:30.754483 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:30.754572 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:30.754614 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0803 22:17:30.829210 20401 new_cluster.go:1168] Cloud Provider ID = aws I0803 22:17:30.829269 20401 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0803 22:17:30.829287 20401 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0803 22:17:30.829298 20401 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0803 22:17:30.829542 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:30.835627 20401 featureflag.go:162] FeatureFlag "Karpenter"=false --- PASS: TestCreateClusterKarpenter (0.08s) === RUN TestParseCloudLabels --- PASS: TestParseCloudLabels (0.00s) === RUN TestConfirmation --- PASS: TestConfirmation (0.00s) === RUN TestMinimal I0803 22:17:30.835838 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:30.835968 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:30.836002 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:30.836026 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:30.836054 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:30.836097 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:30.836115 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:30.836135 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:30.836188 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:30.855353 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:30.858251 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:30.858372 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0803 22:17:33.410833 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:36.115704 20401 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0803 22:17:36.116528 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:36.117527 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:36.118441 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:36.118968 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:17:36.118994 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:36.119072 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:17:36.119107 20401 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0803 22:17:36.123421 20401 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0803 22:17:36.124603 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:36.125619 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:36.128847 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:36.129755 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:36.129897 20401 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0803 22:17:36.130419 20401 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0803 22:17:36.177859 20401 target.go:112] Terraform output is in /tmp/test3787248973/out I0803 22:17:36.181710 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:36.181940 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:36.181996 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:36.182050 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:36.182129 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:36.182177 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:36.182209 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:36.182249 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:36.182279 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:36.207857 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:36.210603 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:36.210704 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0803 22:17:36.251888 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0803 22:17:36.339432 20401 executor.go:111] Tasks: 0 done / 85 total; 43 can run W0803 22:17:36.339913 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0803 22:17:36.340171 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:17:36.340337 20401 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0803 22:17:36.340411 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:36.340373 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0803 22:17:36.340574 20401 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0803 22:17:36.340590 20401 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0803 22:17:36.340594 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:36.340711 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0803 22:17:36.342063 20401 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0803 22:17:36.348635 20401 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0803 22:17:36.349142 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:36.349355 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:36.350236 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:36.352074 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:36.352254 20401 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0803 22:17:36.352512 20401 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0803 22:17:36.356931 20401 target.go:168] Cloudformation output is in /tmp/test1561973133/out --- PASS: TestMinimal (5.53s) === RUN TestMinimal_v1_23 I0803 22:17:36.368726 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:36.369022 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:36.369075 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:36.369123 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:36.369177 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:36.369257 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:36.369290 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:36.369335 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:36.369367 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:36.397479 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* I0803 22:17:36.399351 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:36.399420 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:37.602242 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:37.733030 20401 executor.go:111] Tasks: 0 done / 90 total; 45 can run I0803 22:17:37.734111 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:17:37.734184 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:37.734379 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0803 22:17:37.734398 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:37.734722 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:37.737645 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:37.739969 20401 executor.go:111] Tasks: 45 done / 90 total; 19 can run I0803 22:17:37.747406 20401 executor.go:111] Tasks: 64 done / 90 total; 24 can run I0803 22:17:37.748522 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:37.748815 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:37.751978 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:37.753443 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:37.753526 20401 executor.go:111] Tasks: 88 done / 90 total; 2 can run I0803 22:17:37.754015 20401 executor.go:111] Tasks: 90 done / 90 total; 0 can run I0803 22:17:37.800973 20401 target.go:112] Terraform output is in /tmp/test946983075/out I0803 22:17:37.803415 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:37.803557 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:37.803596 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:37.803628 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:37.803662 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:37.803691 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:37.803708 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:37.803732 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:37.803750 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:37.817323 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:37.818932 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:37.818991 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0803 22:17:37.858382 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0803 22:17:37.953081 20401 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0803 22:17:37.954331 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:37.953375 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0803 22:17:37.954513 20401 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0803 22:17:37.954606 20401 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0803 22:17:37.954571 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:37.954613 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:17:37.954671 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:37.954834 20401 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0803 22:17:37.954727 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } I0803 22:17:37.958875 20401 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0803 22:17:37.961981 20401 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0803 22:17:37.962203 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:37.962412 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:37.963563 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:37.963656 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:37.963887 20401 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0803 22:17:37.964207 20401 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0803 22:17:37.967118 20401 target.go:168] Cloudformation output is in /tmp/test3099633199/out --- PASS: TestMinimal_v1_23 (1.61s) === RUN TestMinimal_v1_24 I0803 22:17:37.977389 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:37.977603 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:37.977643 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:37.977702 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:37.977774 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:37.977898 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:37.977973 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:37.978054 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:37.978113 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:38.005321 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0803 22:17:38.008422 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:38.008521 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:39.469967 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:39.647936 20401 executor.go:111] Tasks: 0 done / 91 total; 46 can run W0803 22:17:39.648480 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:39.648685 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:39.648919 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:17:39.648962 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:39.649108 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:17:39.649118 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:39.655286 20401 executor.go:111] Tasks: 46 done / 91 total; 19 can run I0803 22:17:39.661164 20401 executor.go:111] Tasks: 65 done / 91 total; 24 can run I0803 22:17:39.661881 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:39.662128 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:39.663075 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:39.664753 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:39.664873 20401 executor.go:111] Tasks: 89 done / 91 total; 2 can run I0803 22:17:39.665175 20401 executor.go:111] Tasks: 91 done / 91 total; 0 can run I0803 22:17:39.717740 20401 target.go:112] Terraform output is in /tmp/test3059960329/out I0803 22:17:39.721991 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:39.722249 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:39.722312 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:39.722368 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:39.722427 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:39.722477 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:39.722510 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:39.722578 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:39.722609 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:39.745991 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:39.748856 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:39.748981 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0803 22:17:39.791029 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0803 22:17:39.889410 20401 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0803 22:17:39.889904 20401 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0803 22:17:39.889938 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:39.890036 20401 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0803 22:17:39.890166 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:17:39.890275 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:39.890368 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0803 22:17:39.890459 20401 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0803 22:17:39.890488 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:39.890632 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0803 22:17:39.892077 20401 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0803 22:17:39.900257 20401 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0803 22:17:39.900618 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:39.900875 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:39.901800 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:39.903403 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:39.903626 20401 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0803 22:17:39.904056 20401 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0803 22:17:39.907065 20401 target.go:168] Cloudformation output is in /tmp/test1543978846/out --- PASS: TestMinimal_v1_24 (1.94s) === RUN TestNvidia I0803 22:17:39.917542 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:39.917789 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:39.917846 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:39.917898 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:39.917956 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:39.918035 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:39.918069 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:39.918115 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:39.918146 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:39.942239 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:39.945094 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:39.945193 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:39.984613 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:40.091585 20401 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0803 22:17:40.092415 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:40.092525 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:40.094131 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:17:40.094167 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:40.094269 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0803 22:17:40.094434 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:40.096192 20401 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0803 22:17:40.102115 20401 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0803 22:17:40.102449 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:40.102791 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:40.105687 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:40.108914 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:40.109063 20401 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0803 22:17:40.109406 20401 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0803 22:17:40.153838 20401 target.go:112] Terraform output is in /tmp/test1920358747/out I0803 22:17:40.157919 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:40.158140 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:40.158195 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:40.158247 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:40.158300 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:40.158341 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:40.158372 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:40.158409 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:40.158440 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:40.182840 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:40.185762 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:40.185857 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:40.224893 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0803 22:17:40.330688 20401 executor.go:111] Tasks: 0 done / 86 total; 44 can run W0803 22:17:40.331207 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:17:40.331481 20401 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0803 22:17:40.331667 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0803 22:17:40.331812 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:17:40.331845 20401 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0803 22:17:40.332467 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:40.334139 20401 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0803 22:17:40.334179 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:40.334329 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0803 22:17:40.336378 20401 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0803 22:17:40.342220 20401 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0803 22:17:40.342549 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:40.342771 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:40.343793 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:40.345613 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:40.345770 20401 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0803 22:17:40.346057 20401 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0803 22:17:40.349912 20401 target.go:168] Cloudformation output is in /tmp/test1512537410/out --- PASS: TestNvidia (0.45s) === RUN TestMinimalGossip I0803 22:17:40.362621 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:40.362868 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:40.362922 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:40.362972 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:40.363027 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:40.363109 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:40.363139 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:40.363186 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:40.363216 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:40.388994 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:40.389095 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:40.428263 20401 apply_cluster.go:467] Gossip DNS: skipping DNS validation I0803 22:17:40.532323 20401 executor.go:111] Tasks: 0 done / 84 total; 42 can run W0803 22:17:40.532655 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:40.532905 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:40.533515 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:40.535890 20401 executor.go:111] Tasks: 42 done / 84 total; 19 can run I0803 22:17:40.541148 20401 executor.go:111] Tasks: 61 done / 84 total; 21 can run I0803 22:17:40.542489 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:40.542868 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:40.545403 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:40.548190 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:40.548319 20401 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0803 22:17:40.548627 20401 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0803 22:17:40.597010 20401 target.go:112] Terraform output is in /tmp/test1636004775/out --- PASS: TestMinimalGossip (0.24s) === RUN TestMinimalGossipIRSA I0803 22:17:40.600931 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:40.601190 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:40.601251 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:40.601305 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:40.601363 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:40.601455 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:40.601486 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:40.601528 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:40.601562 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:40.633692 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:40.633795 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:40.672532 20401 apply_cluster.go:467] Gossip DNS: skipping DNS validation I0803 22:17:40.785596 20401 executor.go:111] Tasks: 0 done / 87 total; 44 can run W0803 22:17:40.786117 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:40.786339 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:40.786503 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:40.792685 20401 executor.go:111] Tasks: 44 done / 87 total; 20 can run I0803 22:17:40.797972 20401 executor.go:111] Tasks: 64 done / 87 total; 21 can run I0803 22:17:40.798607 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:40.798964 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:40.800112 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:40.801113 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:40.801233 20401 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0803 22:17:40.801559 20401 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0803 22:17:40.837834 20401 target.go:112] Terraform output is in /tmp/test305866745/out --- PASS: TestMinimalGossipIRSA (0.24s) === RUN TestMinimalGCE I0803 22:17:40.842551 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:40.844460 20401 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* W0803 22:17:41.735576 20401 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0803 22:17:41.735753 20401 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0803 22:17:41.736197 20401 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0803 22:17:41.736323 20401 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0803 22:17:41.736589 20401 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0803 22:17:41.736646 20401 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0803 22:17:41.736764 20401 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0803 22:17:41.738363 20401 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0803 22:17:41.741438 20401 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0803 22:17:41.746594 20401 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0803 22:17:41.753136 20401 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0803 22:17:41.753517 20401 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0803 22:17:41.785469 20401 target.go:112] Terraform output is in /tmp/test3887946652/out --- PASS: TestMinimalGCE (0.95s) === RUN TestMinimalGCEPrivate I0803 22:17:41.789432 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:41.791236 20401 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* W0803 22:17:42.599318 20401 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0803 22:17:42.599456 20401 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0803 22:17:42.599632 20401 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W0803 22:17:42.599723 20401 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/main" W0803 22:17:42.599772 20401 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/events" W0803 22:17:42.599821 20401 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W0803 22:17:42.599921 20401 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0803 22:17:42.601709 20401 executor.go:111] Tasks: 0 done / 66 total; 40 can run I0803 22:17:42.604419 20401 executor.go:111] Tasks: 40 done / 66 total; 18 can run I0803 22:17:42.609941 20401 executor.go:111] Tasks: 58 done / 66 total; 4 can run I0803 22:17:42.616790 20401 executor.go:111] Tasks: 62 done / 66 total; 3 can run I0803 22:17:42.618157 20401 executor.go:111] Tasks: 65 done / 66 total; 1 can run I0803 22:17:42.618353 20401 executor.go:111] Tasks: 66 done / 66 total; 0 can run I0803 22:17:42.651494 20401 target.go:112] Terraform output is in /tmp/test3101727509/out --- PASS: TestMinimalGCEPrivate (0.87s) === RUN TestMinimalGCEInternalLoadBalancer I0803 22:17:42.655324 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:42.657254 20401 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* W0803 22:17:42.830837 20401 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0803 22:17:42.830914 20401 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0803 22:17:42.831043 20401 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/main" W0803 22:17:42.831082 20401 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/events" W0803 22:17:42.831109 20401 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W0803 22:17:42.831144 20401 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W0803 22:17:42.831216 20401 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0803 22:17:42.831265 20401 autoscalinggroup.go:284] Not hooking the instance group manager up to anything. I0803 22:17:42.833071 20401 executor.go:111] Tasks: 0 done / 68 total; 41 can run I0803 22:17:42.836420 20401 executor.go:111] Tasks: 41 done / 68 total; 17 can run W0803 22:17:42.844454 20401 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0803 22:17:42.844499 20401 executor.go:111] Tasks: 58 done / 68 total; 4 can run I0803 22:17:42.847054 20401 executor.go:111] Tasks: 62 done / 68 total; 3 can run I0803 22:17:42.848591 20401 executor.go:111] Tasks: 65 done / 68 total; 2 can run I0803 22:17:42.849012 20401 executor.go:111] Tasks: 67 done / 68 total; 1 can run I0803 22:17:42.849260 20401 executor.go:111] Tasks: 68 done / 68 total; 0 can run I0803 22:17:42.882615 20401 target.go:112] Terraform output is in /tmp/test2189135988/out --- PASS: TestMinimalGCEInternalLoadBalancer (0.23s) === RUN TestMinimalGCELongClusterName I0803 22:17:42.886397 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:42.888195 20401 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* W0803 22:17:43.120900 20401 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0803 22:17:43.121011 20401 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0803 22:17:43.121143 20401 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W0803 22:17:43.121183 20401 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W0803 22:17:43.121212 20401 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0803 22:17:43.121248 20401 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0803 22:17:43.121327 20401 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0803 22:17:43.123065 20401 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0803 22:17:43.126132 20401 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0803 22:17:43.135999 20401 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0803 22:17:43.138992 20401 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0803 22:17:43.139206 20401 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0803 22:17:43.172057 20401 target.go:112] Terraform output is in /tmp/test358719646/out --- PASS: TestMinimalGCELongClusterName (0.29s) === RUN TestMinimalGCEInternalLoadBalancerLongClusterName I0803 22:17:43.176170 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:43.178057 20401 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* W0803 22:17:43.408158 20401 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0803 22:17:43.408257 20401 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0803 22:17:43.408409 20401 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W0803 22:17:43.408448 20401 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W0803 22:17:43.408477 20401 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0803 22:17:43.408513 20401 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0803 22:17:43.408584 20401 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0803 22:17:43.408638 20401 autoscalinggroup.go:284] Not hooking the instance group manager up to anything. I0803 22:17:43.410429 20401 executor.go:111] Tasks: 0 done / 67 total; 42 can run I0803 22:17:43.414706 20401 executor.go:111] Tasks: 42 done / 67 total; 16 can run W0803 22:17:43.421835 20401 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0803 22:17:43.421874 20401 executor.go:111] Tasks: 58 done / 67 total; 5 can run I0803 22:17:43.425096 20401 executor.go:111] Tasks: 63 done / 67 total; 2 can run I0803 22:17:43.425328 20401 executor.go:111] Tasks: 65 done / 67 total; 1 can run I0803 22:17:43.425578 20401 executor.go:111] Tasks: 66 done / 67 total; 1 can run I0803 22:17:43.425737 20401 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0803 22:17:43.457161 20401 target.go:112] Terraform output is in /tmp/test3166611327/out --- PASS: TestMinimalGCEInternalLoadBalancerLongClusterName (0.28s) === RUN TestHA I0803 22:17:43.461244 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:43.461519 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:43.461597 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:43.461681 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:43.461767 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:43.461874 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:43.461930 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:43.461997 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:43.462044 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:43.494598 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:43.497403 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:43.497511 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:43.497590 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:43.497678 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:43.536423 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:43.636831 20401 executor.go:111] Tasks: 0 done / 103 total; 49 can run W0803 22:17:43.637305 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:43.637462 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:43.637573 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:43.638472 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:43.638538 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:17:43.638577 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:43.639033 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:17:43.639785 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.ha.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:43.644528 20401 executor.go:111] Tasks: 49 done / 103 total; 23 can run I0803 22:17:43.652642 20401 executor.go:111] Tasks: 72 done / 103 total; 27 can run I0803 22:17:43.653185 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:43.653444 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:43.653679 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:43.654652 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:43.656902 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:43.659523 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:43.659721 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:43.660059 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:43.660193 20401 executor.go:111] Tasks: 99 done / 103 total; 4 can run I0803 22:17:43.660606 20401 executor.go:111] Tasks: 103 done / 103 total; 0 can run I0803 22:17:43.733814 20401 target.go:112] Terraform output is in /tmp/test1881369414/out --- PASS: TestHA (0.28s) === RUN TestHighAvailabilityGCE I0803 22:17:43.738730 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:43.740857 20401 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* W0803 22:17:43.927422 20401 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0803 22:17:43.927515 20401 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0803 22:17:43.927649 20401 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/main" W0803 22:17:43.927687 20401 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/events" W0803 22:17:43.927714 20401 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W0803 22:17:43.927749 20401 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W0803 22:17:43.927813 20401 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0803 22:17:43.927894 20401 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0803 22:17:43.927970 20401 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0803 22:17:43.930162 20401 executor.go:111] Tasks: 0 done / 79 total; 44 can run I0803 22:17:43.933504 20401 executor.go:111] Tasks: 44 done / 79 total; 20 can run I0803 22:17:43.944506 20401 executor.go:111] Tasks: 64 done / 79 total; 7 can run I0803 22:17:43.947480 20401 executor.go:111] Tasks: 71 done / 79 total; 5 can run I0803 22:17:43.948722 20401 executor.go:111] Tasks: 76 done / 79 total; 3 can run I0803 22:17:43.948989 20401 executor.go:111] Tasks: 79 done / 79 total; 0 can run I0803 22:17:43.988384 20401 target.go:112] Terraform output is in /tmp/test1072709844/out --- PASS: TestHighAvailabilityGCE (0.25s) === RUN TestComplex I0803 22:17:43.993511 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:43.993757 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:43.993819 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:43.993966 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:43.994018 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:43.994064 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:43.994096 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:43.994135 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:43.994164 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:44.030359 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:44.034247 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:44.034362 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:44.073373 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:44.173976 20401 executor.go:111] Tasks: 0 done / 116 total; 47 can run W0803 22:17:44.174607 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:44.174758 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:44.174840 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:17:44.174879 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:44.175013 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:17:44.176770 20401 executor.go:111] Tasks: 47 done / 116 total; 27 can run I0803 22:17:44.186346 20401 executor.go:111] Tasks: 74 done / 116 total; 39 can run I0803 22:17:44.186766 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:44.186975 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:44.188003 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:44.189526 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:44.189725 20401 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0803 22:17:44.190447 20401 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0803 22:17:44.262310 20401 target.go:112] Terraform output is in /tmp/test3978959325/out I0803 22:17:44.266193 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:44.266437 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:44.266502 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:44.266559 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:44.266617 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:44.266663 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:44.266695 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:44.266738 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:44.266768 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:44.298957 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:44.302832 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:44.302934 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:44.342231 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0803 22:17:44.443179 20401 executor.go:111] Tasks: 0 done / 116 total; 47 can run I0803 22:17:44.443917 20401 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0803 22:17:44.444039 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:17:44.443967 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:44.444103 20401 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0803 22:17:44.444214 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:17:44.446007 20401 executor.go:111] Tasks: 47 done / 116 total; 27 can run I0803 22:17:44.455299 20401 executor.go:111] Tasks: 74 done / 116 total; 39 can run I0803 22:17:44.455742 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:44.455956 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:44.456845 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:44.458806 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:44.458944 20401 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0803 22:17:44.459241 20401 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0803 22:17:44.463843 20401 target.go:168] Cloudformation output is in /tmp/test3973766592/out I0803 22:17:44.476841 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:44.477090 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:44.477144 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:44.477190 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:44.477301 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:44.477352 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:44.477381 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:44.477419 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:44.477449 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:44.514486 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:44.518388 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:44.518498 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:44.556822 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:44.661104 20401 executor.go:111] Tasks: 0 done / 116 total; 47 can run I0803 22:17:44.661855 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0803 22:17:44.661939 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:44.661900 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:44.662079 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0803 22:17:44.662041 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:44.667012 20401 executor.go:111] Tasks: 47 done / 116 total; 27 can run I0803 22:17:44.673658 20401 executor.go:111] Tasks: 74 done / 116 total; 39 can run I0803 22:17:44.674206 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:44.674447 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:44.675362 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:44.676955 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:44.677079 20401 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0803 22:17:44.677445 20401 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0803 22:17:44.752229 20401 target.go:112] Terraform output is in /tmp/test1525877855/out --- PASS: TestComplex (0.76s) === RUN TestCompress I0803 22:17:44.756605 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:44.756873 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:44.756941 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:44.757018 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:44.757108 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:44.757219 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:44.757281 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:44.757346 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:44.757398 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:44.780733 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:44.783738 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:44.783864 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:44.822901 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:44.924145 20401 executor.go:111] Tasks: 0 done / 84 total; 42 can run W0803 22:17:44.925054 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:44.925059 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:44.925248 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:17:44.925285 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:44.925401 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:17:44.927519 20401 executor.go:111] Tasks: 42 done / 84 total; 19 can run I0803 22:17:44.935719 20401 executor.go:111] Tasks: 61 done / 84 total; 21 can run I0803 22:17:44.936318 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:44.936463 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:44.939641 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:44.941244 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:44.941365 20401 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0803 22:17:44.941725 20401 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0803 22:17:44.996219 20401 target.go:112] Terraform output is in /tmp/test240761723/out --- PASS: TestCompress (0.24s) === RUN TestExternalPolicies I0803 22:17:45.000434 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:45.000691 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:45.000751 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:45.000804 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:45.000857 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:45.000955 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:45.000991 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:45.001028 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:45.001056 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:45.029220 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:45.032574 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:45.032669 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:45.072203 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:45.171824 20401 executor.go:111] Tasks: 0 done / 98 total; 45 can run W0803 22:17:45.172303 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:45.172562 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externalpolicies.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:45.172797 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:17:45.173174 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:45.173271 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0803 22:17:45.172750 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:45.178940 20401 executor.go:111] Tasks: 45 done / 98 total; 20 can run I0803 22:17:45.185971 20401 executor.go:111] Tasks: 65 done / 98 total; 30 can run I0803 22:17:45.186347 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:45.186566 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:45.187710 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:45.189018 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:45.189223 20401 executor.go:111] Tasks: 95 done / 98 total; 3 can run I0803 22:17:45.189726 20401 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0803 22:17:45.255733 20401 target.go:112] Terraform output is in /tmp/test2906419353/out --- PASS: TestExternalPolicies (0.26s) === RUN TestMinimalIPv6 I0803 22:17:45.259766 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:45.260028 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:45.260086 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:45.260138 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:45.260192 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:45.260286 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:45.260317 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:45.260354 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:45.260381 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:45.285541 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:45.288808 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:45.288906 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:45.328289 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:45.496355 20401 executor.go:111] Tasks: 0 done / 104 total; 45 can run W0803 22:17:45.496629 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:45.496704 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:17:45.496735 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:45.496792 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:45.497169 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:45.496837 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:17:45.504183 20401 executor.go:111] Tasks: 45 done / 104 total; 21 can run I0803 22:17:45.509849 20401 executor.go:111] Tasks: 66 done / 104 total; 30 can run I0803 22:17:45.510290 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:45.510578 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:45.511819 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:45.513461 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:45.513589 20401 executor.go:111] Tasks: 96 done / 104 total; 5 can run I0803 22:17:45.514276 20401 executor.go:111] Tasks: 101 done / 104 total; 3 can run I0803 22:17:45.514609 20401 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0803 22:17:45.577927 20401 target.go:112] Terraform output is in /tmp/test1341959156/out I0803 22:17:45.582287 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:45.582521 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:45.582584 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:45.582637 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:45.582694 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:45.582740 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:45.582772 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:45.582814 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:45.582840 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:45.608056 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:45.611357 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:45.611457 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:45.650549 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0803 22:17:45.826575 20401 executor.go:111] Tasks: 0 done / 104 total; 45 can run I0803 22:17:45.827340 20401 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0803 22:17:45.827416 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:45.827397 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:45.827534 20401 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0803 22:17:45.827606 20401 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key W0803 22:17:45.827651 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0803 22:17:45.827658 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:17:45.827637 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:45.827793 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0803 22:17:45.831778 20401 executor.go:111] Tasks: 45 done / 104 total; 21 can run I0803 22:17:45.839225 20401 executor.go:111] Tasks: 66 done / 104 total; 30 can run I0803 22:17:45.839842 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:45.840086 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:45.841322 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:45.842445 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:45.842585 20401 executor.go:111] Tasks: 96 done / 104 total; 5 can run I0803 22:17:45.843010 20401 executor.go:111] Tasks: 101 done / 104 total; 3 can run I0803 22:17:45.843335 20401 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0803 22:17:45.847526 20401 target.go:168] Cloudformation output is in /tmp/test1074152484/out --- PASS: TestMinimalIPv6 (0.60s) === RUN TestMinimalIPv6Private I0803 22:17:45.859341 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:45.859578 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:45.859632 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:45.859679 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:45.859732 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:45.859817 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:45.859849 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:45.859889 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:45.859919 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:45.893043 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* I0803 22:17:45.896643 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:45.896743 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:45.937994 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:46.115831 20401 executor.go:111] Tasks: 0 done / 122 total; 46 can run I0803 22:17:46.116162 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:17:46.116204 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:46.116298 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0803 22:17:46.117436 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:46.117504 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:46.117746 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:46.123376 20401 executor.go:111] Tasks: 46 done / 122 total; 23 can run I0803 22:17:46.129469 20401 executor.go:111] Tasks: 69 done / 122 total; 36 can run I0803 22:17:46.130009 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:46.130231 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:46.131878 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:46.133073 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:46.133211 20401 executor.go:111] Tasks: 105 done / 122 total; 11 can run I0803 22:17:46.133916 20401 executor.go:111] Tasks: 116 done / 122 total; 6 can run I0803 22:17:46.135511 20401 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0803 22:17:46.210046 20401 target.go:112] Terraform output is in /tmp/test2818592888/out --- PASS: TestMinimalIPv6Private (0.36s) === RUN TestMinimalIPv6Calico I0803 22:17:46.215017 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:46.215287 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:46.215348 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:46.215406 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:46.215482 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:46.215598 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:46.215630 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:46.215670 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:46.215701 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:46.245350 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0803 22:17:46.248598 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:46.248702 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:47.222962 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:47.632787 20401 executor.go:111] Tasks: 0 done / 107 total; 47 can run W0803 22:17:47.633186 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:47.633387 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:47.633507 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:47.633792 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:17:47.633846 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:47.633986 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:17:47.637361 20401 executor.go:111] Tasks: 47 done / 107 total; 21 can run I0803 22:17:47.646444 20401 executor.go:111] Tasks: 68 done / 107 total; 31 can run I0803 22:17:47.646734 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:47.647141 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:47.648121 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:47.649913 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:47.650055 20401 executor.go:111] Tasks: 99 done / 107 total; 5 can run I0803 22:17:47.650581 20401 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0803 22:17:47.650877 20401 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0803 22:17:47.716470 20401 target.go:112] Terraform output is in /tmp/test429801578/out I0803 22:17:47.721721 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:47.721955 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:47.722012 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:47.722064 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:47.722139 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:47.722188 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:47.722219 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:47.722259 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:47.722290 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:47.751348 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0803 22:17:47.755589 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:47.755705 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:47.795977 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0803 22:17:48.205998 20401 executor.go:111] Tasks: 0 done / 107 total; 47 can run W0803 22:17:48.206381 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0803 22:17:48.206735 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:17:48.206757 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:48.206839 20401 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0803 22:17:48.206897 20401 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0803 22:17:48.206918 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:48.207016 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0803 22:17:48.206878 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:48.207205 20401 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0803 22:17:48.211194 20401 executor.go:111] Tasks: 47 done / 107 total; 21 can run I0803 22:17:48.219025 20401 executor.go:111] Tasks: 68 done / 107 total; 31 can run I0803 22:17:48.219716 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:48.219960 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:48.221519 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:48.222782 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:48.222950 20401 executor.go:111] Tasks: 99 done / 107 total; 5 can run I0803 22:17:48.223350 20401 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0803 22:17:48.223670 20401 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0803 22:17:48.227673 20401 target.go:168] Cloudformation output is in /tmp/test2174632696/out --- PASS: TestMinimalIPv6Calico (2.02s) === RUN TestMinimalIPv6Cilium I0803 22:17:48.239406 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:48.239634 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:48.239685 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:48.239732 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:48.239787 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:48.239869 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:48.239900 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:48.239936 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:48.239965 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:48.268261 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:48.272359 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:48.272470 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:48.313246 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:48.526909 20401 executor.go:111] Tasks: 0 done / 105 total; 46 can run I0803 22:17:48.527386 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:48.529552 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:17:48.529610 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:48.529778 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0803 22:17:48.530470 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:48.532505 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:48.534749 20401 executor.go:111] Tasks: 46 done / 105 total; 21 can run I0803 22:17:48.540418 20401 executor.go:111] Tasks: 67 done / 105 total; 30 can run I0803 22:17:48.541096 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:48.541411 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:48.543404 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:48.547786 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:48.547953 20401 executor.go:111] Tasks: 97 done / 105 total; 5 can run I0803 22:17:48.548643 20401 executor.go:111] Tasks: 102 done / 105 total; 3 can run I0803 22:17:48.548970 20401 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0803 22:17:48.609913 20401 target.go:112] Terraform output is in /tmp/test2515381963/out I0803 22:17:48.614231 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:48.614463 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:48.614525 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:48.614581 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:48.614638 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:48.614683 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:48.614713 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:48.614750 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:48.614783 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:48.643316 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:48.646919 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:48.647028 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:48.686509 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0803 22:17:48.901416 20401 executor.go:111] Tasks: 0 done / 105 total; 46 can run I0803 22:17:48.903041 20401 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0803 22:17:48.903098 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:48.903224 20401 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0803 22:17:48.903991 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:48.904194 20401 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0803 22:17:48.904219 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:48.904329 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } W0803 22:17:48.906299 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0803 22:17:48.906538 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:17:48.908445 20401 executor.go:111] Tasks: 46 done / 105 total; 21 can run I0803 22:17:48.914359 20401 executor.go:111] Tasks: 67 done / 105 total; 30 can run I0803 22:17:48.914926 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:48.915141 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:48.916711 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:48.921844 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:48.922139 20401 executor.go:111] Tasks: 97 done / 105 total; 5 can run I0803 22:17:48.922610 20401 executor.go:111] Tasks: 102 done / 105 total; 3 can run I0803 22:17:48.922981 20401 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0803 22:17:48.927102 20401 target.go:168] Cloudformation output is in /tmp/test2928386643/out --- PASS: TestMinimalIPv6Cilium (0.70s) === RUN TestMinimalWarmPool I0803 22:17:48.938707 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:48.938966 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:48.939022 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:48.939074 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:48.939131 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:48.939216 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:48.939246 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:48.939285 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:48.939316 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:48.963772 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:48.967148 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:48.967241 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:49.007282 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:49.203279 20401 executor.go:111] Tasks: 0 done / 88 total; 45 can run I0803 22:17:49.203627 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-warmpool.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:49.204569 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:49.207420 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:49.207902 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:17:49.207951 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:49.208052 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:17:49.210733 20401 executor.go:111] Tasks: 45 done / 88 total; 19 can run I0803 22:17:49.216317 20401 executor.go:111] Tasks: 64 done / 88 total; 21 can run I0803 22:17:49.216770 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:49.217030 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:49.218200 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:49.219372 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:49.219501 20401 executor.go:111] Tasks: 85 done / 88 total; 2 can run I0803 22:17:49.219858 20401 executor.go:111] Tasks: 87 done / 88 total; 1 can run I0803 22:17:49.220171 20401 executor.go:111] Tasks: 88 done / 88 total; 0 can run I0803 22:17:49.270004 20401 target.go:112] Terraform output is in /tmp/test1155823936/out --- PASS: TestMinimalWarmPool (0.34s) === RUN TestMinimalEtcd I0803 22:17:49.274623 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:49.274864 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:49.274919 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:49.275023 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:49.275075 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:49.275121 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:49.275150 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:49.275198 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:49.275242 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:49.304556 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:49.307617 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:49.307721 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:49.347176 20401 zones.go:93] ListHostedZonesPages { } W0803 22:17:49.432788 20401 model.go:247] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W0803 22:17:49.432858 20401 model.go:347] overriding log level in manifest (embedded etcd manifest), new level is 10 W0803 22:17:49.434655 20401 model.go:247] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W0803 22:17:49.434823 20401 model.go:480] overloading ENV var in manifest (embedded etcd manifest) with ETCD_MANAGER_HOURLY_BACKUPS_RETENTION=1d W0803 22:17:49.434874 20401 model.go:480] overloading ENV var in manifest (embedded etcd manifest) with ETCD_MANAGER_DAILY_BACKUPS_RETENTION=30d ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0803 22:17:49.447166 20401 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0803 22:17:49.447452 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:49.447655 20401 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0803 22:17:49.447685 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:49.447799 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0803 22:17:49.447936 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:17:49.447835 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-etcd.example.com" },{ Key: "KubernetesCluster", Value: "minimal-etcd.example.com" },{ Key: "kubernetes.io/cluster/minimal-etcd.example.com", Value: "owned" }] }] } I0803 22:17:49.448249 20401 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0803 22:17:49.448275 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:49.448381 20401 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0803 22:17:49.450510 20401 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0803 22:17:49.455869 20401 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0803 22:17:49.456078 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:49.456330 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:49.458950 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:49.462318 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:49.462477 20401 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0803 22:17:49.462762 20401 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0803 22:17:49.465866 20401 target.go:168] Cloudformation output is in /tmp/test2467204221/out --- PASS: TestMinimalEtcd (0.20s) === RUN TestMinimalGp3 I0803 22:17:49.476436 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:49.476670 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:49.476724 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:49.476771 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:49.476823 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:49.476906 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:49.476936 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:49.476975 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:49.477005 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:49.501267 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:49.504561 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:49.504654 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:49.543777 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:49.646057 20401 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0803 22:17:49.646380 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:17:49.646425 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:49.646530 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0803 22:17:49.646576 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:49.646844 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:49.646871 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:49.648957 20401 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0803 22:17:49.657427 20401 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0803 22:17:49.657758 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:49.657939 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:49.658885 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:49.660770 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:49.660886 20401 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0803 22:17:49.661192 20401 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0803 22:17:49.714093 20401 target.go:112] Terraform output is in /tmp/test3353693379/out I0803 22:17:49.717967 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:49.718192 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:49.718254 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:49.718306 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:49.718360 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:49.718408 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:49.718452 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:49.718495 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:49.718525 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:49.742694 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:49.745196 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:49.745282 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:49.786695 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0803 22:17:49.886568 20401 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0803 22:17:49.886864 20401 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0803 22:17:49.886910 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:49.887014 20401 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0803 22:17:49.887126 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:49.887261 20401 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0803 22:17:49.887277 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:49.887400 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } W0803 22:17:49.887499 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0803 22:17:49.887569 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:17:49.889451 20401 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0803 22:17:49.897641 20401 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0803 22:17:49.898094 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:49.898235 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:49.899376 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:49.900556 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:49.900682 20401 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0803 22:17:49.900924 20401 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0803 22:17:49.903874 20401 target.go:168] Cloudformation output is in /tmp/test2649252118/out --- PASS: TestMinimalGp3 (0.44s) === RUN TestMinimalLongClusterName I0803 22:17:49.914022 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:49.914254 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:49.914306 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:49.914351 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:49.914404 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:49.914485 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:49.914519 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:49.914563 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:49.914592 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:49.938661 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:49.941338 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:49.941429 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0803 22:17:49.980348 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:50.081805 20401 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0803 22:17:50.082372 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:50.082436 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:50.082776 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0803 22:17:50.082866 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:50.082802 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:50.082921 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:17:50.085804 20401 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0803 22:17:50.093988 20401 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0803 22:17:50.094487 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.094697 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.095845 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.097255 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.097373 20401 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0803 22:17:50.097703 20401 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0803 22:17:50.155306 20401 target.go:112] Terraform output is in /tmp/test914246916/out I0803 22:17:50.159463 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:50.159691 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:50.159754 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:50.159806 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:50.159864 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:50.159912 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:50.159942 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:50.159984 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:50.160014 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:50.182774 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:50.185256 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.185345 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0803 22:17:50.223781 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0803 22:17:50.322313 20401 executor.go:111] Tasks: 0 done / 85 total; 43 can run W0803 22:17:50.322636 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:17:50.322768 20401 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0803 22:17:50.322812 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:50.322904 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:50.322959 20401 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0803 22:17:50.323058 20401 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0803 22:17:50.323079 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:50.323212 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:17:50.323179 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "this.is.truly.a.really.really.long.cluster-name.minimal.example.com" },{ Key: "KubernetesCluster", Value: "this.is.truly.a.really.really.long.cluster-name.minimal.example.com" },{ Key: "kubernetes.io/cluster/this.is.truly.a.really.really.long.cluster-name.minimal.example.com", Value: "owned" }] }] } I0803 22:17:50.325707 20401 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0803 22:17:50.334665 20401 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0803 22:17:50.335211 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.335458 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.336633 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.337553 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.337689 20401 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0803 22:17:50.337961 20401 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0803 22:17:50.341433 20401 target.go:168] Cloudformation output is in /tmp/test1558463383/out --- PASS: TestMinimalLongClusterName (0.44s) === RUN TestExistingIAMCloudformation I0803 22:17:50.352303 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:50.352540 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:50.352598 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:50.352689 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:50.352741 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:50.352786 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:50.352815 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:50.352852 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:50.352882 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:50.377113 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:50.380276 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.380365 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.419136 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0803 22:17:50.520688 20401 executor.go:111] Tasks: 0 done / 75 total; 41 can run I0803 22:17:50.521100 20401 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0803 22:17:50.521142 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:50.521251 20401 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0803 22:17:50.521393 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:50.521463 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0803 22:17:50.521482 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0803 22:17:50.521581 20401 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0803 22:17:50.521611 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:50.521752 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0803 22:17:50.523939 20401 executor.go:111] Tasks: 41 done / 75 total; 11 can run I0803 22:17:50.529237 20401 executor.go:111] Tasks: 52 done / 75 total; 21 can run I0803 22:17:50.529758 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.530038 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.530780 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.532257 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.532392 20401 executor.go:111] Tasks: 73 done / 75 total; 2 can run I0803 22:17:50.532647 20401 executor.go:111] Tasks: 75 done / 75 total; 0 can run I0803 22:17:50.534825 20401 target.go:168] Cloudformation output is in /tmp/test2162659123/out --- PASS: TestExistingIAMCloudformation (0.19s) === RUN TestExistingSG I0803 22:17:50.542971 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:50.543354 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:50.543409 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:50.543455 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:50.543505 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:50.543587 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:50.543618 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:50.543658 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:50.543688 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:50.580843 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:50.584049 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.584149 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.584211 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.584270 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.623270 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:50.728514 20401 executor.go:111] Tasks: 0 done / 138 total; 49 can run I0803 22:17:50.729151 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existingsg.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:50.729353 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0803 22:17:50.729381 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:50.729432 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:50.729396 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:50.729535 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0803 22:17:50.729562 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:50.729627 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:50.731490 20401 executor.go:111] Tasks: 49 done / 138 total; 26 can run I0803 22:17:50.741440 20401 executor.go:111] Tasks: 75 done / 138 total; 58 can run I0803 22:17:50.741989 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.742229 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.742410 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.742672 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.743620 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.744413 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.744622 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.744974 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.745106 20401 executor.go:111] Tasks: 133 done / 138 total; 5 can run I0803 22:17:50.745618 20401 executor.go:111] Tasks: 138 done / 138 total; 0 can run I0803 22:17:50.839167 20401 target.go:112] Terraform output is in /tmp/test435735843/out --- PASS: TestExistingSG (0.31s) === RUN TestBastionAdditionalUserData I0803 22:17:50.851638 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:50.851919 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:50.851983 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:50.852036 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:50.852093 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:50.852210 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:50.852243 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:50.852285 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:50.852317 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:50.882787 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:50.885907 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.886011 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.886077 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:50.924821 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:51.026523 20401 executor.go:111] Tasks: 0 done / 119 total; 48 can run W0803 22:17:51.027374 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:51.027420 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.bastionuserdata.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:51.027588 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:51.027742 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0803 22:17:51.027786 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:51.027772 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:51.027881 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:17:51.033623 20401 executor.go:111] Tasks: 48 done / 119 total; 29 can run I0803 22:17:51.039699 20401 executor.go:111] Tasks: 77 done / 119 total; 36 can run I0803 22:17:51.040282 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:51.040578 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:51.040849 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:51.041520 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:51.041786 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:51.043175 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:51.043307 20401 executor.go:111] Tasks: 113 done / 119 total; 5 can run I0803 22:17:51.043764 20401 executor.go:111] Tasks: 118 done / 119 total; 1 can run I0803 22:17:51.043924 20401 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0803 22:17:51.123294 20401 target.go:112] Terraform output is in /tmp/test3887809795/out --- PASS: TestBastionAdditionalUserData (0.28s) === RUN TestPrivateWeave I0803 22:17:51.128272 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:51.128513 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:51.128566 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:51.128617 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:51.128672 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:51.128759 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:51.128814 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:51.128854 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:51.128883 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:51.156197 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:51.159195 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:51.159288 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:51.159351 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:51.198548 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:51.317477 20401 executor.go:111] Tasks: 0 done / 117 total; 47 can run I0803 22:17:51.318077 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateweave.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:51.318375 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0803 22:17:51.318502 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:51.318503 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:51.319098 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:51.318416 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:51.319415 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:17:51.324828 20401 executor.go:111] Tasks: 47 done / 117 total; 28 can run I0803 22:17:51.330168 20401 executor.go:111] Tasks: 75 done / 117 total; 36 can run I0803 22:17:51.330622 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:51.331009 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:51.331258 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:51.331399 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:51.332570 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:51.333365 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:51.333494 20401 executor.go:111] Tasks: 111 done / 117 total; 5 can run I0803 22:17:51.334003 20401 executor.go:111] Tasks: 116 done / 117 total; 1 can run I0803 22:17:51.334274 20401 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0803 22:17:51.408323 20401 target.go:112] Terraform output is in /tmp/test595643078/out --- PASS: TestPrivateWeave (0.28s) === RUN TestPrivateFlannel I0803 22:17:51.412710 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:51.413008 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:51.413076 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:51.413142 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:51.413218 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:51.413324 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:51.413357 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:51.413411 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:51.413446 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:51.441329 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:51.444358 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:51.444455 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:51.444529 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:51.483192 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:51.603812 20401 executor.go:111] Tasks: 0 done / 117 total; 47 can run W0803 22:17:51.604093 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:51.604168 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:51.604396 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateflannel.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:51.604655 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:51.604678 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:17:51.604698 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:51.604759 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:17:51.607059 20401 executor.go:111] Tasks: 47 done / 117 total; 28 can run I0803 22:17:51.612533 20401 executor.go:111] Tasks: 75 done / 117 total; 36 can run I0803 22:17:51.612842 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:51.613975 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:51.614200 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:51.614481 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:51.617987 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:51.619105 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:51.619241 20401 executor.go:111] Tasks: 111 done / 117 total; 5 can run I0803 22:17:51.619733 20401 executor.go:111] Tasks: 116 done / 117 total; 1 can run I0803 22:17:51.619884 20401 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0803 22:17:51.691104 20401 target.go:112] Terraform output is in /tmp/test855990165/out --- PASS: TestPrivateFlannel (0.28s) === RUN TestPrivateCalico I0803 22:17:51.695481 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:51.695724 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:51.695781 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:51.695839 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:51.695893 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:51.695981 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:51.696012 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:51.696050 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:51.696081 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:51.727230 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0803 22:17:51.730667 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:51.730772 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:51.730840 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:51.773064 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:52.191581 20401 executor.go:111] Tasks: 0 done / 124 total; 50 can run W0803 22:17:52.192093 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:52.192238 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:17:52.192290 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:52.192388 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0803 22:17:52.192561 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:52.192639 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:52.192905 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:52.198747 20401 executor.go:111] Tasks: 50 done / 124 total; 28 can run I0803 22:17:52.204243 20401 executor.go:111] Tasks: 78 done / 124 total; 40 can run I0803 22:17:52.204479 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:52.204820 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:52.204989 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:52.205202 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:52.205783 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:52.207336 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:52.207452 20401 executor.go:111] Tasks: 118 done / 124 total; 5 can run I0803 22:17:52.207921 20401 executor.go:111] Tasks: 123 done / 124 total; 1 can run I0803 22:17:52.208131 20401 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0803 22:17:52.285772 20401 target.go:112] Terraform output is in /tmp/test316585060/out I0803 22:17:52.291500 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:52.291751 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:52.291814 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:52.291868 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:52.291927 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:52.291976 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:52.292007 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:52.292049 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:52.292077 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:52.321318 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0803 22:17:52.324488 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:52.324606 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:52.324672 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:52.363479 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0803 22:17:52.799223 20401 executor.go:111] Tasks: 0 done / 124 total; 50 can run W0803 22:17:52.799777 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:17:52.799893 20401 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0803 22:17:52.799931 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:52.800084 20401 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0803 22:17:52.800350 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0803 22:17:52.800389 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:17:52.800401 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:52.800557 20401 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0803 22:17:52.800579 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:52.800664 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0803 22:17:52.807050 20401 executor.go:111] Tasks: 50 done / 124 total; 28 can run I0803 22:17:52.812986 20401 executor.go:111] Tasks: 78 done / 124 total; 40 can run I0803 22:17:52.813399 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:52.813763 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:52.813948 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:52.814020 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:52.815122 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:52.817006 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:52.817146 20401 executor.go:111] Tasks: 118 done / 124 total; 5 can run I0803 22:17:52.817608 20401 executor.go:111] Tasks: 123 done / 124 total; 1 can run I0803 22:17:52.817752 20401 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0803 22:17:52.822140 20401 target.go:168] Cloudformation output is in /tmp/test1501132840/out --- PASS: TestPrivateCalico (1.14s) === RUN TestPrivateCilium I0803 22:17:52.835242 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:52.835498 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:52.835552 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:52.835600 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:52.835653 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:52.835736 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:52.835766 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:52.835810 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:52.835840 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:52.868329 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:52.871555 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:52.871651 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:52.871719 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:52.913587 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:53.061319 20401 executor.go:111] Tasks: 0 done / 117 total; 47 can run W0803 22:17:53.061631 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:53.061856 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:53.061988 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:53.062220 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:53.062523 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:17:53.062566 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:53.062705 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:17:53.068487 20401 executor.go:111] Tasks: 47 done / 117 total; 28 can run I0803 22:17:53.073930 20401 executor.go:111] Tasks: 75 done / 117 total; 36 can run I0803 22:17:53.074589 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:53.074824 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:53.075009 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:53.075348 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:53.076372 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:53.077307 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:53.077432 20401 executor.go:111] Tasks: 111 done / 117 total; 5 can run I0803 22:17:53.078137 20401 executor.go:111] Tasks: 116 done / 117 total; 1 can run I0803 22:17:53.078308 20401 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0803 22:17:53.154617 20401 target.go:112] Terraform output is in /tmp/test3025853190/out I0803 22:17:53.159036 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:53.159266 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:53.159330 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:53.159383 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:53.159440 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:53.159490 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:53.159520 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:53.159562 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:53.159592 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:53.187066 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:53.190290 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:53.190390 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:53.190452 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:53.229226 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0803 22:17:53.374204 20401 executor.go:111] Tasks: 0 done / 117 total; 47 can run W0803 22:17:53.374789 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0803 22:17:53.374930 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0803 22:17:53.374993 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:17:53.375121 20401 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0803 22:17:53.375189 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:53.375153 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0803 22:17:53.375315 20401 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0803 22:17:53.375335 20401 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0803 22:17:53.375334 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:53.375457 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecilium.example.com" },{ Key: "KubernetesCluster", Value: "privatecilium.example.com" },{ Key: "kubernetes.io/cluster/privatecilium.example.com", Value: "owned" }] }] } I0803 22:17:53.377004 20401 executor.go:111] Tasks: 47 done / 117 total; 28 can run I0803 22:17:53.382493 20401 executor.go:111] Tasks: 75 done / 117 total; 36 can run I0803 22:17:53.382744 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:53.383247 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:53.383492 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:53.383663 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:53.384288 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:53.385143 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:53.385304 20401 executor.go:111] Tasks: 111 done / 117 total; 5 can run I0803 22:17:53.385724 20401 executor.go:111] Tasks: 116 done / 117 total; 1 can run I0803 22:17:53.385883 20401 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0803 22:17:53.392543 20401 target.go:168] Cloudformation output is in /tmp/test181893133/out --- PASS: TestPrivateCilium (0.57s) === RUN TestPrivateCilium2 I0803 22:17:53.406339 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:53.406573 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:53.406626 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:53.406675 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:53.406728 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:53.406808 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:53.406840 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:53.406879 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:53.406909 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:53.434282 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.19.16 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.19.16 ********************************************************************************* I0803 22:17:53.437473 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:53.437558 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:53.437619 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:54.553910 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:54.693967 20401 executor.go:111] Tasks: 0 done / 117 total; 47 can run W0803 22:17:54.694316 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:54.694323 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:54.694653 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:17:54.694693 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:54.694777 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0803 22:17:54.694976 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:54.695032 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:54.696617 20401 executor.go:111] Tasks: 47 done / 117 total; 28 can run I0803 22:17:54.700407 20401 executor.go:111] Tasks: 75 done / 117 total; 36 can run I0803 22:17:54.700572 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:54.700902 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:54.701022 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:54.701088 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:54.701679 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:54.703328 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:54.703459 20401 executor.go:111] Tasks: 111 done / 117 total; 5 can run I0803 22:17:54.703895 20401 executor.go:111] Tasks: 116 done / 117 total; 1 can run I0803 22:17:54.704056 20401 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0803 22:17:54.780711 20401 target.go:112] Terraform output is in /tmp/test848337617/out I0803 22:17:54.784986 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:54.785225 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:54.785287 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:54.785339 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:54.785396 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:54.785463 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:54.785498 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:54.785538 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:54.785569 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:54.813255 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.19.16 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.19.16 ********************************************************************************* I0803 22:17:54.816604 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:54.816696 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:54.816756 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:54.857898 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0803 22:17:54.997568 20401 executor.go:111] Tasks: 0 done / 117 total; 47 can run W0803 22:17:54.997964 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:17:54.998049 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:54.998198 20401 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0803 22:17:54.998218 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:54.998342 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:17:54.998323 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecilium.example.com" },{ Key: "KubernetesCluster", Value: "privatecilium.example.com" },{ Key: "kubernetes.io/cluster/privatecilium.example.com", Value: "owned" }] }] } W0803 22:17:54.998556 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:17:54.998590 20401 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0803 22:17:54.998622 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:54.998745 20401 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0803 22:17:55.001212 20401 executor.go:111] Tasks: 47 done / 117 total; 28 can run I0803 22:17:55.006776 20401 executor.go:111] Tasks: 75 done / 117 total; 36 can run I0803 22:17:55.007279 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:55.007573 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:55.007850 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:55.008006 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:55.008746 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:55.009401 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:55.009571 20401 executor.go:111] Tasks: 111 done / 117 total; 5 can run I0803 22:17:55.010759 20401 executor.go:111] Tasks: 116 done / 117 total; 1 can run I0803 22:17:55.011253 20401 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0803 22:17:55.017930 20401 target.go:168] Cloudformation output is in /tmp/test3202867417/out --- PASS: TestPrivateCilium2 (1.63s) === RUN TestPrivateCiliumAdvanced I0803 22:17:55.031759 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:55.032002 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:55.032056 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:55.032149 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:55.032243 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:55.032372 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:55.032409 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:55.032447 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:55.032476 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:55.064745 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:55.068150 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:55.068246 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:55.068309 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:55.109008 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:55.262634 20401 executor.go:111] Tasks: 0 done / 123 total; 53 can run I0803 22:17:55.263454 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0803 22:17:55.263557 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:55.263501 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:55.263603 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:17:55.263629 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:55.263760 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:55.263687 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:55.267160 20401 executor.go:111] Tasks: 53 done / 123 total; 28 can run I0803 22:17:55.277012 20401 executor.go:111] Tasks: 81 done / 123 total; 36 can run I0803 22:17:55.277504 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:55.277888 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:55.278155 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:55.278383 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:55.279448 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:55.280378 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:55.280508 20401 executor.go:111] Tasks: 117 done / 123 total; 5 can run I0803 22:17:55.280979 20401 executor.go:111] Tasks: 122 done / 123 total; 1 can run I0803 22:17:55.281164 20401 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0803 22:17:55.358871 20401 target.go:112] Terraform output is in /tmp/test3679810930/out I0803 22:17:55.363568 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:55.363803 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:55.363861 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:55.363915 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:55.363969 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:55.364016 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:55.364045 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:55.364085 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:55.364114 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:55.396915 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:55.399880 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:55.399964 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:55.400020 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:55.439306 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0803 22:17:55.593188 20401 executor.go:111] Tasks: 0 done / 123 total; 53 can run I0803 22:17:55.593739 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:55.593905 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0803 22:17:55.593934 20401 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0803 22:17:55.593959 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:55.594140 20401 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0803 22:17:55.594112 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "privateciliumadvanced.example.com" },{ Key: "kubernetes.io/cluster/privateciliumadvanced.example.com", Value: "owned" },{ Key: "Name", Value: "privateciliumadvanced.example.com" }] }] } W0803 22:17:55.596939 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:17:55.594176 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:55.597527 20401 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0803 22:17:55.597689 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:17:55.601513 20401 executor.go:111] Tasks: 53 done / 123 total; 28 can run I0803 22:17:55.608415 20401 executor.go:111] Tasks: 81 done / 123 total; 36 can run I0803 22:17:55.608914 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:55.609082 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:55.609137 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:55.609390 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:55.611086 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:55.612364 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:55.612514 20401 executor.go:111] Tasks: 117 done / 123 total; 5 can run I0803 22:17:55.613069 20401 executor.go:111] Tasks: 122 done / 123 total; 1 can run I0803 22:17:55.613261 20401 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0803 22:17:55.617663 20401 target.go:168] Cloudformation output is in /tmp/test2414609424/out --- PASS: TestPrivateCiliumAdvanced (0.60s) === RUN TestPrivateCanal I0803 22:17:55.636279 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:55.636527 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:55.636580 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:55.636629 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:55.636680 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:55.636769 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:55.636801 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:55.636841 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:55.636869 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:55.664003 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.10 ********************************************************************************* I0803 22:17:55.667020 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:55.667107 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:55.667168 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:56.794104 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:57.205326 20401 executor.go:111] Tasks: 0 done / 118 total; 48 can run I0803 22:17:57.205719 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:17:57.205768 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:57.205918 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0803 22:17:57.206275 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:57.206369 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:57.206386 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecanal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:57.206460 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:57.212615 20401 executor.go:111] Tasks: 48 done / 118 total; 28 can run I0803 22:17:57.218592 20401 executor.go:111] Tasks: 76 done / 118 total; 36 can run I0803 22:17:57.219116 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:57.219279 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:57.219536 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:57.219763 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:57.220593 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:57.221897 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:57.222032 20401 executor.go:111] Tasks: 112 done / 118 total; 5 can run I0803 22:17:57.222497 20401 executor.go:111] Tasks: 117 done / 118 total; 1 can run I0803 22:17:57.222699 20401 executor.go:111] Tasks: 118 done / 118 total; 0 can run I0803 22:17:57.298415 20401 target.go:112] Terraform output is in /tmp/test2314659901/out --- PASS: TestPrivateCanal (1.67s) === RUN TestPrivateKopeio I0803 22:17:57.304156 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:57.304408 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:57.304478 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:57.304530 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:57.304584 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:57.304674 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:57.304704 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:57.304744 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:57.304774 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:57.338222 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:57.342123 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:57.342235 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:57.342302 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:57.381706 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:57.497324 20401 executor.go:111] Tasks: 0 done / 123 total; 47 can run I0803 22:17:57.497678 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:17:57.497714 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:57.497820 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:17:57.497918 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:57.498220 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:57.498454 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:57.498536 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:57.502430 20401 executor.go:111] Tasks: 47 done / 123 total; 31 can run I0803 22:17:57.509444 20401 executor.go:111] Tasks: 78 done / 123 total; 39 can run I0803 22:17:57.509835 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:57.510210 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:57.510500 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:57.510687 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:57.511325 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:57.512194 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:57.512320 20401 executor.go:111] Tasks: 117 done / 123 total; 6 can run I0803 22:17:57.512767 20401 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0803 22:17:57.594656 20401 target.go:112] Terraform output is in /tmp/test1035161280/out --- PASS: TestPrivateKopeio (0.29s) === RUN TestUnmanaged I0803 22:17:57.599160 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:57.599409 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:57.599470 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:57.599525 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:57.599583 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:57.599678 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:57.599709 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:57.599747 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:57.599777 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:57.629670 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:57.632810 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:57.632902 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:57.632966 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:57.671360 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:57.773842 20401 executor.go:111] Tasks: 0 done / 105 total; 45 can run I0803 22:17:57.774145 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:17:57.774226 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.unmanaged.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:57.774182 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:57.774416 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0803 22:17:57.774525 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:57.774663 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:57.774870 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:57.778503 20401 executor.go:111] Tasks: 45 done / 105 total; 25 can run I0803 22:17:57.786542 20401 executor.go:111] Tasks: 70 done / 105 total; 31 can run I0803 22:17:57.786969 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:57.787097 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:57.787303 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:57.787467 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:57.788228 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:57.789694 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:57.789836 20401 executor.go:111] Tasks: 101 done / 105 total; 4 can run I0803 22:17:57.790264 20401 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0803 22:17:57.860791 20401 target.go:112] Terraform output is in /tmp/test1122927349/out --- PASS: TestUnmanaged (0.27s) === RUN TestPrivateSharedSubnet I0803 22:17:57.864973 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:57.865231 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:57.865297 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:57.865351 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:57.865408 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:57.865529 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:57.865563 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:57.865604 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:57.865637 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:57.894484 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:57.897637 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:57.897734 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:57.897796 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:57.936764 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:58.021319 20401 executor.go:111] Tasks: 0 done / 104 total; 45 can run W0803 22:17:58.021897 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:58.022017 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:58.022023 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0803 22:17:58.022052 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:58.022046 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:58.022134 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:17:58.022082 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:58.024730 20401 executor.go:111] Tasks: 45 done / 104 total; 24 can run I0803 22:17:58.025324 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0803 22:17:58.035330 20401 executor.go:111] Tasks: 69 done / 104 total; 31 can run I0803 22:17:58.035772 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:58.035954 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:58.036019 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:58.036277 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:58.037418 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:58.037879 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:58.037974 20401 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0803 22:17:58.038356 20401 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0803 22:17:58.105353 20401 target.go:112] Terraform output is in /tmp/test821178807/out --- PASS: TestPrivateSharedSubnet (0.24s) === RUN TestPrivateSharedIP I0803 22:17:58.109839 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:58.110088 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:58.110144 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:58.110268 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:58.110320 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:58.110364 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:58.110394 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:58.110431 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:58.110460 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:58.142720 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:58.145993 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:58.146092 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:58.146154 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:58.185421 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:58.288155 20401 executor.go:111] Tasks: 0 done / 113 total; 45 can run W0803 22:17:58.288566 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:58.288857 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:58.289117 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:17:58.289148 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:58.289239 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0803 22:17:58.289324 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:58.289397 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:58.290864 20401 executor.go:111] Tasks: 45 done / 113 total; 26 can run I0803 22:17:58.291036 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0803 22:17:58.296127 20401 executor.go:111] Tasks: 71 done / 113 total; 36 can run I0803 22:17:58.296565 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:58.296858 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:58.297061 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:58.297341 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:58.297933 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:58.298815 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:58.298935 20401 executor.go:111] Tasks: 107 done / 113 total; 5 can run I0803 22:17:58.299382 20401 executor.go:111] Tasks: 112 done / 113 total; 1 can run I0803 22:17:58.299547 20401 executor.go:111] Tasks: 113 done / 113 total; 0 can run I0803 22:17:58.373294 20401 target.go:112] Terraform output is in /tmp/test1678575034/out I0803 22:17:58.377409 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:58.377633 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:58.377695 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:58.378143 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:58.378201 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:58.378247 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:58.378278 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:58.378320 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:58.378348 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:58.406250 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:58.409328 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:58.409425 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:58.409488 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:58.448992 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0803 22:17:58.550865 20401 executor.go:111] Tasks: 0 done / 113 total; 45 can run W0803 22:17:58.551804 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0803 22:17:58.551896 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:17:58.551975 20401 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0803 22:17:58.552018 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:17:58.552021 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:58.552198 20401 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0803 22:17:58.552203 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:58.552392 20401 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0803 22:17:58.552421 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:58.552532 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0803 22:17:58.555554 20401 executor.go:111] Tasks: 45 done / 113 total; 26 can run I0803 22:17:58.557551 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0803 22:17:58.563581 20401 executor.go:111] Tasks: 71 done / 113 total; 36 can run I0803 22:17:58.564360 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:58.564588 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:58.564762 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:58.564870 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:58.566015 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:58.566654 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:17:58.566904 20401 executor.go:111] Tasks: 107 done / 113 total; 5 can run I0803 22:17:58.567516 20401 executor.go:111] Tasks: 112 done / 113 total; 1 can run I0803 22:17:58.567700 20401 executor.go:111] Tasks: 113 done / 113 total; 0 can run I0803 22:17:58.571709 20401 target.go:168] Cloudformation output is in /tmp/test2113248266/out --- PASS: TestPrivateSharedIP (0.47s) === RUN TestPrivateDns1 I0803 22:17:58.584662 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:58.584892 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:58.584947 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:58.584995 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:58.585051 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:58.585133 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:58.585163 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:58.585199 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:58.585229 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:58.620101 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:58.620204 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:58.620269 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:58.777606 20401 executor.go:111] Tasks: 0 done / 117 total; 46 can run W0803 22:17:58.778057 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:58.779691 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:58.779930 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:58.780051 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns1.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:58.785527 20401 executor.go:111] Tasks: 46 done / 117 total; 24 can run I0803 22:17:58.786032 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "internal.example.com" I0803 22:17:58.786073 20401 zones.go:40] GetHostedZone { Id: "/hostedzone/Z2AFAKE1ZON3NO" } I0803 22:17:58.786160 20401 dnszone.go:249] Existing zone "internal.example.com." found; will configure TF to reuse I0803 22:17:58.786179 20401 dnszone.go:272] No association between VPC "privatedns1.example.com" and zone "internal.example.com."; adding I0803 22:17:58.786271 20401 executor.go:111] Tasks: 70 done / 117 total; 35 can run I0803 22:17:58.786882 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:58.787090 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:58.790303 20401 executor.go:111] Tasks: 105 done / 117 total; 5 can run W0803 22:17:58.790425 20401 api.go:53] PageSize not implemented W0803 22:17:58.790462 20401 api.go:53] PageSize not implemented W0803 22:17:58.790465 20401 bootstrapscript.go:89] Task did not have an address: *awstasks.ClassicLoadBalancer {"Name":"api.privatedns1.example.com","Lifecycle":"Sync","LoadBalancerName":"api-privatedns1-example-c-lq96ht","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-us-test-1a.privatedns1.example.com","ShortName":"utility-us-test-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"privatedns1.example.com","Lifecycle":"Sync","ID":null,"CIDR":"172.20.0.0/16","AmazonIPv6":true,"IPv6CIDR":null,"EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"AssociateExtraCIDRBlocks":null},"AmazonIPv6CIDR":null,"AvailabilityZone":"us-test-1a","CIDR":"172.20.4.0/22","IPv6CIDR":null,"ResourceBasedNaming":null,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"utility-us-test-1a.privatedns1.example.com","Owner":"John Doe","SubnetType":"Utility","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.privatedns1.example.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"privatedns1.example.com","Lifecycle":"Sync","ID":null,"CIDR":"172.20.0.0/16","AmazonIPv6":true,"IPv6CIDR":null,"EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"AssociateExtraCIDRBlocks":null},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"api-elb.privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":{"EmitInterval":null,"Enabled":false,"S3BucketName":null,"S3BucketPrefix":null},"ConnectionDraining":{"Enabled":true,"Timeout":300},"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":{"Enabled":false},"SSLCertificateID":"","Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"api.privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"ForAPIServer":true,"Shared":null} W0803 22:17:58.790499 20401 bootstrapscript.go:89] Task did not have an address: *awstasks.ClassicLoadBalancer {"Name":"api.privatedns1.example.com","Lifecycle":"Sync","LoadBalancerName":"api-privatedns1-example-c-lq96ht","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-us-test-1a.privatedns1.example.com","ShortName":"utility-us-test-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"privatedns1.example.com","Lifecycle":"Sync","ID":null,"CIDR":"172.20.0.0/16","AmazonIPv6":true,"IPv6CIDR":null,"EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"AssociateExtraCIDRBlocks":null},"AmazonIPv6CIDR":null,"AvailabilityZone":"us-test-1a","CIDR":"172.20.4.0/22","IPv6CIDR":null,"ResourceBasedNaming":null,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"utility-us-test-1a.privatedns1.example.com","Owner":"John Doe","SubnetType":"Utility","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.privatedns1.example.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"privatedns1.example.com","Lifecycle":"Sync","ID":null,"CIDR":"172.20.0.0/16","AmazonIPv6":true,"IPv6CIDR":null,"EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"AssociateExtraCIDRBlocks":null},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"api-elb.privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":{"EmitInterval":null,"Enabled":false,"S3BucketName":null,"S3BucketPrefix":null},"ConnectionDraining":{"Enabled":true,"Timeout":300},"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":{"Enabled":false},"SSLCertificateID":"","Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"api.privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"ForAPIServer":true,"Shared":null} I0803 22:17:58.796753 20401 executor.go:111] Tasks: 110 done / 117 total; 5 can run I0803 22:17:58.796937 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:58.797163 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:58.797994 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:58.799817 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:58.799951 20401 executor.go:111] Tasks: 115 done / 117 total; 2 can run I0803 22:17:58.800279 20401 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0803 22:17:58.885122 20401 target.go:112] Terraform output is in /tmp/test4169804157/out --- PASS: TestPrivateDns1 (0.30s) === RUN TestPrivateDns2 I0803 22:17:58.889551 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:58.889816 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:58.889880 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:58.889936 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:58.889994 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:58.890087 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:58.890117 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:58.890156 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:58.890188 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:58.921277 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:58.921379 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:58.921448 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:59.059412 20401 executor.go:111] Tasks: 0 done / 113 total; 44 can run W0803 22:17:59.059828 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:59.060134 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:59.060147 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:59.060536 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns2.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:59.064375 20401 executor.go:111] Tasks: 44 done / 113 total; 22 can run I0803 22:17:59.065687 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "private.example.com" I0803 22:17:59.065764 20401 zones.go:40] GetHostedZone { Id: "/hostedzone/Z3AFAKE1ZOMORE" } I0803 22:17:59.065928 20401 dnszone.go:249] Existing zone "private.example.com." found; will configure TF to reuse I0803 22:17:59.065669 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0803 22:17:59.066008 20401 dnszone.go:263] VPC "vpc-12345678" already associated with zone "private.example.com." I0803 22:17:59.067305 20401 executor.go:111] Tasks: 66 done / 113 total; 35 can run I0803 22:17:59.068040 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:59.068299 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:59.071035 20401 executor.go:111] Tasks: 101 done / 113 total; 5 can run W0803 22:17:59.071130 20401 api.go:53] PageSize not implemented W0803 22:17:59.071183 20401 api.go:53] PageSize not implemented W0803 22:17:59.071158 20401 bootstrapscript.go:89] Task did not have an address: *awstasks.ClassicLoadBalancer {"Name":"api.privatedns2.example.com","Lifecycle":"Sync","LoadBalancerName":"api-privatedns2-example-c-6jft30","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-us-test-1a.privatedns2.example.com","ShortName":"utility-us-test-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"privatedns2.example.com","Lifecycle":"Sync","ID":"vpc-12345678","CIDR":"172.20.0.0/16","AmazonIPv6":null,"IPv6CIDR":null,"EnableDNSHostnames":null,"EnableDNSSupport":true,"Shared":true,"Tags":null,"AssociateExtraCIDRBlocks":null},"AmazonIPv6CIDR":null,"AvailabilityZone":"us-test-1a","CIDR":"172.20.4.0/22","IPv6CIDR":null,"ResourceBasedNaming":null,"Shared":false,"Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"utility-us-test-1a.privatedns2.example.com","SubnetType":"Utility","kubernetes.io/cluster/privatedns2.example.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.privatedns2.example.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"privatedns2.example.com","Lifecycle":"Sync","ID":"vpc-12345678","CIDR":"172.20.0.0/16","AmazonIPv6":null,"IPv6CIDR":null,"EnableDNSHostnames":null,"EnableDNSSupport":true,"Shared":true,"Tags":null,"AssociateExtraCIDRBlocks":null},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"api-elb.privatedns2.example.com","kubernetes.io/cluster/privatedns2.example.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":{"EmitInterval":null,"Enabled":false,"S3BucketName":null,"S3BucketPrefix":null},"ConnectionDraining":{"Enabled":true,"Timeout":300},"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":{"Enabled":false},"SSLCertificateID":"","Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"api.privatedns2.example.com","kubernetes.io/cluster/privatedns2.example.com":"owned"},"ForAPIServer":true,"Shared":null} W0803 22:17:59.071210 20401 bootstrapscript.go:89] Task did not have an address: *awstasks.ClassicLoadBalancer {"Name":"api.privatedns2.example.com","Lifecycle":"Sync","LoadBalancerName":"api-privatedns2-example-c-6jft30","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-us-test-1a.privatedns2.example.com","ShortName":"utility-us-test-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"privatedns2.example.com","Lifecycle":"Sync","ID":"vpc-12345678","CIDR":"172.20.0.0/16","AmazonIPv6":null,"IPv6CIDR":null,"EnableDNSHostnames":null,"EnableDNSSupport":true,"Shared":true,"Tags":null,"AssociateExtraCIDRBlocks":null},"AmazonIPv6CIDR":null,"AvailabilityZone":"us-test-1a","CIDR":"172.20.4.0/22","IPv6CIDR":null,"ResourceBasedNaming":null,"Shared":false,"Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"utility-us-test-1a.privatedns2.example.com","SubnetType":"Utility","kubernetes.io/cluster/privatedns2.example.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.privatedns2.example.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"privatedns2.example.com","Lifecycle":"Sync","ID":"vpc-12345678","CIDR":"172.20.0.0/16","AmazonIPv6":null,"IPv6CIDR":null,"EnableDNSHostnames":null,"EnableDNSSupport":true,"Shared":true,"Tags":null,"AssociateExtraCIDRBlocks":null},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"api-elb.privatedns2.example.com","kubernetes.io/cluster/privatedns2.example.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":{"EmitInterval":null,"Enabled":false,"S3BucketName":null,"S3BucketPrefix":null},"ConnectionDraining":{"Enabled":true,"Timeout":300},"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":{"Enabled":false},"SSLCertificateID":"","Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"api.privatedns2.example.com","kubernetes.io/cluster/privatedns2.example.com":"owned"},"ForAPIServer":true,"Shared":null} I0803 22:17:59.076104 20401 executor.go:111] Tasks: 106 done / 113 total; 5 can run I0803 22:17:59.076374 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:59.076630 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:59.077783 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:59.078866 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:59.079068 20401 executor.go:111] Tasks: 111 done / 113 total; 2 can run I0803 22:17:59.079435 20401 executor.go:111] Tasks: 113 done / 113 total; 0 can run I0803 22:17:59.152847 20401 target.go:112] Terraform output is in /tmp/test4002740644/out --- PASS: TestPrivateDns2 (0.27s) === RUN TestDiscoveryFeatureGate I0803 22:17:59.157104 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:59.157357 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:59.157415 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:59.157470 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:59.157525 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:59.157612 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:59.157642 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:59.157679 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:59.157708 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:59.182817 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.19.16 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.19.16 ********************************************************************************* I0803 22:17:59.186101 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:59.186199 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:59.229482 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:59.339571 20401 executor.go:111] Tasks: 0 done / 90 total; 46 can run I0803 22:17:59.340019 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:17:59.340199 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0803 22:17:59.340317 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:59.340243 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:59.340381 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0803 22:17:59.340607 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:59.342613 20401 executor.go:111] Tasks: 46 done / 90 total; 21 can run I0803 22:17:59.347690 20401 executor.go:111] Tasks: 67 done / 90 total; 21 can run I0803 22:17:59.348172 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:59.348454 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:59.351308 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:59.354168 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:59.354344 20401 executor.go:111] Tasks: 88 done / 90 total; 2 can run I0803 22:17:59.354685 20401 executor.go:111] Tasks: 90 done / 90 total; 0 can run I0803 22:17:59.406979 20401 target.go:112] Terraform output is in /tmp/test3619800468/out --- PASS: TestDiscoveryFeatureGate (0.25s) === RUN TestVFSServiceAccountIssuerDiscovery I0803 22:17:59.411296 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:59.411556 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:59.411616 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:59.411670 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:59.411727 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:59.411820 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:59.411852 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:59.411894 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:59.411926 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:59.440750 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:59.443768 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:59.443868 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:59.484198 20401 zones.go:93] ListHostedZonesPages { } I0803 22:17:59.582520 20401 executor.go:111] Tasks: 0 done / 88 total; 45 can run I0803 22:17:59.583165 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:17:59.583443 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:17:59.583503 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:17:59.583690 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:17:59.583719 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:17:59.583830 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:17:59.585589 20401 executor.go:111] Tasks: 45 done / 88 total; 20 can run I0803 22:17:59.590442 20401 executor.go:111] Tasks: 65 done / 88 total; 21 can run I0803 22:17:59.590749 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:59.591099 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:59.593369 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:59.597240 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:59.597377 20401 executor.go:111] Tasks: 86 done / 88 total; 2 can run I0803 22:17:59.597703 20401 executor.go:111] Tasks: 88 done / 88 total; 0 can run I0803 22:17:59.648027 20401 target.go:112] Terraform output is in /tmp/test1819255916/out --- PASS: TestVFSServiceAccountIssuerDiscovery (0.24s) === RUN TestAWSLBController I0803 22:17:59.652198 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:59.652455 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:59.652517 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:59.652576 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:59.652634 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:59.652726 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:59.652759 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:59.652798 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:59.652831 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:59.681774 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:17:59.684811 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:59.684916 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:17:59.724313 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:00.341940 20401 executor.go:111] Tasks: 0 done / 94 total; 49 can run W0803 22:18:00.342431 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:00.342586 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:18:00.342620 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:00.342717 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:18:00.342955 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:18:00.343064 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:00.350007 20401 executor.go:111] Tasks: 49 done / 94 total; 22 can run I0803 22:18:00.355342 20401 executor.go:111] Tasks: 71 done / 94 total; 21 can run I0803 22:18:00.355633 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:00.355854 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:00.356754 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:00.358231 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:00.358354 20401 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0803 22:18:00.358665 20401 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0803 22:18:00.416230 20401 target.go:112] Terraform output is in /tmp/test657601015/out --- PASS: TestAWSLBController (0.77s) === RUN TestManyAddons I0803 22:18:00.424489 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:00.424745 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:00.424808 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:00.424861 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:00.424939 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:00.425108 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:00.425158 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:00.425233 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:00.425286 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:00.451651 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:00.455274 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:00.455375 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:00.494893 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:01.296154 20401 executor.go:111] Tasks: 0 done / 93 total; 50 can run W0803 22:18:01.296569 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:01.296704 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:18:01.296735 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:01.296816 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0803 22:18:01.296854 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:01.297101 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:01.303414 20401 executor.go:111] Tasks: 50 done / 93 total; 19 can run I0803 22:18:01.309662 20401 executor.go:111] Tasks: 69 done / 93 total; 22 can run I0803 22:18:01.310127 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:01.310363 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:01.311508 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:01.312628 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:01.312746 20401 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0803 22:18:01.313068 20401 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0803 22:18:01.366166 20401 target.go:112] Terraform output is in /tmp/test914841371/out --- PASS: TestManyAddons (0.95s) === RUN TestManyAddonsCCMIRSA I0803 22:18:01.378955 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:01.379219 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:01.379285 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:01.379342 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:01.379400 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:01.379500 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:01.379531 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:01.379570 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:01.379601 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:01.407680 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:01.411210 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:01.411308 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:01.450405 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:02.319360 20401 executor.go:111] Tasks: 0 done / 109 total; 59 can run W0803 22:18:02.319736 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:02.320367 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:18:02.320504 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:02.320553 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:18:02.320600 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:02.320739 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:18:02.326601 20401 executor.go:111] Tasks: 59 done / 109 total; 26 can run I0803 22:18:02.334268 20401 executor.go:111] Tasks: 85 done / 109 total; 22 can run I0803 22:18:02.334745 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:02.334963 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:02.336342 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:02.337739 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:02.337881 20401 executor.go:111] Tasks: 107 done / 109 total; 2 can run I0803 22:18:02.338194 20401 executor.go:111] Tasks: 109 done / 109 total; 0 can run I0803 22:18:02.406403 20401 target.go:112] Terraform output is in /tmp/test1427940886/out --- PASS: TestManyAddonsCCMIRSA (1.04s) === RUN TestManyAddonsCCMIRSA23 I0803 22:18:02.416685 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:02.416939 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:02.417002 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:02.417058 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:02.417112 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:02.417205 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:02.417238 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:02.417278 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:02.417308 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:02.442196 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* I0803 22:18:02.444924 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:02.444999 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:02.486509 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:03.357344 20401 executor.go:111] Tasks: 0 done / 110 total; 60 can run W0803 22:18:03.358007 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:03.358299 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:18:03.358329 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:03.358436 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0803 22:18:03.358548 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:03.360864 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:03.366987 20401 executor.go:111] Tasks: 60 done / 110 total; 26 can run I0803 22:18:03.372372 20401 executor.go:111] Tasks: 86 done / 110 total; 22 can run I0803 22:18:03.372594 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:03.372723 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:03.375642 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:03.379119 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:03.379269 20401 executor.go:111] Tasks: 108 done / 110 total; 2 can run I0803 22:18:03.379660 20401 executor.go:111] Tasks: 110 done / 110 total; 0 can run I0803 22:18:03.446417 20401 target.go:112] Terraform output is in /tmp/test1403452455/out --- PASS: TestManyAddonsCCMIRSA23 (1.04s) === RUN TestManyAddonsCCMIRSA24 I0803 22:18:03.455613 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:03.455867 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:03.455927 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:03.456041 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:03.456095 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:03.456141 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:03.456172 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:03.456216 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:03.456244 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:03.483202 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0803 22:18:03.486239 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:03.486320 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:03.527463 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:04.398562 20401 executor.go:111] Tasks: 0 done / 110 total; 60 can run I0803 22:18:04.399586 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:18:04.399746 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:18:04.399935 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:04.400128 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:18:04.400194 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:04.400344 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:18:04.407841 20401 executor.go:111] Tasks: 60 done / 110 total; 26 can run I0803 22:18:04.413772 20401 executor.go:111] Tasks: 86 done / 110 total; 22 can run I0803 22:18:04.414159 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:04.414385 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:04.415544 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:04.416664 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:04.416860 20401 executor.go:111] Tasks: 108 done / 110 total; 2 can run I0803 22:18:04.417412 20401 executor.go:111] Tasks: 110 done / 110 total; 0 can run I0803 22:18:04.486934 20401 target.go:112] Terraform output is in /tmp/test2377768299/out --- PASS: TestManyAddonsCCMIRSA24 (1.04s) === RUN TestCCM I0803 22:18:04.496828 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:04.497083 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:04.497140 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:04.497192 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:04.497242 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:04.497332 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:04.497366 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:04.497420 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:04.497449 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:04.523930 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:04.527076 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:04.527160 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:04.567994 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:05.372428 20401 executor.go:111] Tasks: 0 done / 94 total; 51 can run I0803 22:18:05.373879 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:18:05.374106 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:18:05.374872 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:05.377238 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:18:05.377276 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:05.377425 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:18:05.383638 20401 executor.go:111] Tasks: 51 done / 94 total; 19 can run I0803 22:18:05.392434 20401 executor.go:111] Tasks: 70 done / 94 total; 22 can run I0803 22:18:05.392660 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:05.392960 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:05.394184 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:05.395099 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:05.395221 20401 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0803 22:18:05.395575 20401 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0803 22:18:05.450970 20401 target.go:112] Terraform output is in /tmp/test2656373987/out --- PASS: TestCCM (0.96s) === RUN TestExternalDNS I0803 22:18:05.459149 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:05.459425 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:05.459491 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:05.459544 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:05.459602 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:05.459698 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:05.459735 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:05.459775 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:05.459807 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:05.483052 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:05.485373 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:05.485479 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:05.525064 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:05.629648 20401 executor.go:111] Tasks: 0 done / 85 total; 43 can run W0803 22:18:05.630430 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:05.630591 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0803 22:18:05.630613 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:05.630639 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:05.630781 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:18:05.630812 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:05.632980 20401 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0803 22:18:05.642972 20401 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0803 22:18:05.643538 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:05.643768 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:05.645020 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:05.646413 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:05.646536 20401 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0803 22:18:05.646826 20401 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0803 22:18:05.701020 20401 target.go:112] Terraform output is in /tmp/test1034691422/out I0803 22:18:05.705043 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:05.705271 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:05.705332 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:05.705385 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:05.705456 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:05.705506 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:05.705538 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:05.705577 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:05.705608 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:05.728987 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:05.731435 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:05.731522 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:05.769972 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0803 22:18:05.871005 20401 executor.go:111] Tasks: 0 done / 85 total; 43 can run W0803 22:18:05.871292 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:18:05.871267 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:18:05.871333 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0803 22:18:05.871396 20401 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0803 22:18:05.871412 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:05.871519 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0803 22:18:05.872028 20401 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0803 22:18:05.872063 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:05.872201 20401 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0803 22:18:05.874343 20401 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0803 22:18:05.884401 20401 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0803 22:18:05.884640 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:05.885022 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:05.885789 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:05.887350 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:05.887482 20401 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0803 22:18:05.887727 20401 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0803 22:18:05.890692 20401 target.go:168] Cloudformation output is in /tmp/test2801869674/out --- PASS: TestExternalDNS (0.44s) === RUN TestExternalDNSIRSA I0803 22:18:05.901408 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:05.901632 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:05.901682 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:05.901728 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:05.901782 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:05.901878 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:05.901911 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:05.901951 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:05.901979 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:05.926799 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:05.929656 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:05.929741 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:05.969379 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:06.086578 20401 executor.go:111] Tasks: 0 done / 90 total; 46 can run I0803 22:18:06.086814 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:06.087176 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0803 22:18:06.089905 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:06.087221 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0803 22:18:06.090453 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:06.090495 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:18:06.094334 20401 executor.go:111] Tasks: 46 done / 90 total; 21 can run I0803 22:18:06.099767 20401 executor.go:111] Tasks: 67 done / 90 total; 21 can run I0803 22:18:06.099967 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.100187 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.101247 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.102505 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.102626 20401 executor.go:111] Tasks: 88 done / 90 total; 2 can run I0803 22:18:06.102956 20401 executor.go:111] Tasks: 90 done / 90 total; 0 can run I0803 22:18:06.159933 20401 target.go:112] Terraform output is in /tmp/test118665149/out --- PASS: TestExternalDNSIRSA (0.26s) === RUN TestKarpenter I0803 22:18:06.165300 20401 featureflag.go:162] FeatureFlag "Karpenter"=true I0803 22:18:06.165340 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:06.165613 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:06.165678 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:06.165731 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:06.165787 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:06.165883 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:06.165915 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:06.165955 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:06.165987 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:06.196047 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:06.199119 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.199209 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.199270 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.238868 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:06.332423 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.332586 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.332699 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.426771 20401 executor.go:111] Tasks: 0 done / 101 total; 50 can run I0803 22:18:06.427286 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:18:06.427484 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:18:06.427934 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:18:06.428938 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:18:06.427887 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:06.427957 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:18:06.430416 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:06.430567 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:18:06.434543 20401 executor.go:111] Tasks: 50 done / 101 total; 24 can run I0803 22:18:06.440035 20401 executor.go:111] Tasks: 74 done / 101 total; 25 can run I0803 22:18:06.440279 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.440526 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.440727 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.440934 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.441485 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.441646 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.441894 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.443389 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.443512 20401 executor.go:111] Tasks: 99 done / 101 total; 2 can run I0803 22:18:06.443800 20401 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0803 22:18:06.515611 20401 target.go:112] Terraform output is in /tmp/test1190214268/out I0803 22:18:06.520848 20401 featureflag.go:162] FeatureFlag "Karpenter"=false --- PASS: TestKarpenter (0.36s) === RUN TestSharedSubnet I0803 22:18:06.521011 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:06.521289 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:06.521351 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:06.521404 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:06.521458 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:06.521551 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:06.521580 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:06.521618 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:06.521649 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:06.545914 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:06.548825 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.548921 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.587771 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:06.686245 20401 executor.go:111] Tasks: 0 done / 78 total; 42 can run W0803 22:18:06.687197 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:06.687198 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0803 22:18:06.687270 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:06.687282 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:06.687253 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:06.687441 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:18:06.690309 20401 executor.go:111] Tasks: 42 done / 78 total; 16 can run I0803 22:18:06.691229 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0803 22:18:06.699229 20401 executor.go:111] Tasks: 58 done / 78 total; 18 can run I0803 22:18:06.699906 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.700185 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.701871 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.703394 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.703511 20401 executor.go:111] Tasks: 76 done / 78 total; 2 can run I0803 22:18:06.703901 20401 executor.go:111] Tasks: 78 done / 78 total; 0 can run I0803 22:18:06.750943 20401 target.go:112] Terraform output is in /tmp/test1930953317/out --- PASS: TestSharedSubnet (0.24s) === RUN TestSharedVPC I0803 22:18:06.758461 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:06.758814 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:06.758873 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:06.758926 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:06.758980 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:06.759182 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:06.759231 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:06.759275 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:06.759306 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:06.783902 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:06.786768 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.786862 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.826411 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:06.927421 20401 executor.go:111] Tasks: 0 done / 82 total; 42 can run W0803 22:18:06.927766 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:06.928166 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0803 22:18:06.928255 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:06.928202 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:06.928306 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:18:06.928340 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:06.930660 20401 executor.go:111] Tasks: 42 done / 82 total; 17 can run I0803 22:18:06.930887 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0803 22:18:06.938959 20401 executor.go:111] Tasks: 59 done / 82 total; 21 can run I0803 22:18:06.939278 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.939459 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.940380 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.941700 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:06.941817 20401 executor.go:111] Tasks: 80 done / 82 total; 2 can run I0803 22:18:06.942127 20401 executor.go:111] Tasks: 82 done / 82 total; 0 can run I0803 22:18:06.989608 20401 target.go:112] Terraform output is in /tmp/test3687919502/out --- PASS: TestSharedVPC (0.24s) === RUN TestExistingIAM I0803 22:18:06.994449 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:06.994707 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:06.994765 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:06.994817 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:06.994870 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:06.994979 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:06.995010 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:06.995047 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:06.995080 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:07.030759 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:07.033954 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:07.034061 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:07.034127 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:07.034197 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:07.073154 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:07.172481 20401 executor.go:111] Tasks: 0 done / 93 total; 47 can run I0803 22:18:07.173252 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existing-iam.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:07.173426 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0803 22:18:07.173445 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:18:07.173458 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:07.173456 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:07.173548 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0803 22:18:07.173598 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:18:07.173653 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:07.175831 20401 executor.go:111] Tasks: 47 done / 93 total; 15 can run I0803 22:18:07.181297 20401 executor.go:111] Tasks: 62 done / 93 total; 27 can run I0803 22:18:07.181606 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:07.181828 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:07.182024 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:07.183314 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:07.187950 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:07.188380 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:07.188504 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:07.188722 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:07.188864 20401 executor.go:111] Tasks: 89 done / 93 total; 4 can run I0803 22:18:07.189217 20401 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0803 22:18:07.260105 20401 target.go:112] Terraform output is in /tmp/test3622700001/out --- PASS: TestExistingIAM (0.27s) === RUN TestPhaseNetwork I0803 22:18:07.264271 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:07.264526 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:07.264588 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:07.264642 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:07.264698 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:07.264743 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:07.264773 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:07.264813 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:07.264842 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:07.295512 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:07.299099 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:07.299205 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:07.299270 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:07.338837 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:07.443620 20401 executor.go:111] Tasks: 0 done / 116 total; 46 can run I0803 22:18:07.443921 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.lifecyclephases.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:07.444061 20401 executor.go:111] Tasks: 46 done / 116 total; 28 can run I0803 22:18:07.444590 20401 executor.go:111] Tasks: 74 done / 116 total; 36 can run I0803 22:18:07.445122 20401 executor.go:111] Tasks: 110 done / 116 total; 5 can run I0803 22:18:07.445355 20401 executor.go:111] Tasks: 115 done / 116 total; 1 can run I0803 22:18:07.445533 20401 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0803 22:18:07.453948 20401 target.go:112] Terraform output is in /tmp/test1321933655/out --- PASS: TestPhaseNetwork (0.19s) === RUN TestExternalLoadBalancer I0803 22:18:07.454702 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:07.454938 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:07.454995 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:07.455045 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:07.455099 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:07.455190 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:07.455223 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:07.455265 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:07.455293 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:07.480020 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:07.482741 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:07.482836 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:07.523252 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:07.621755 20401 executor.go:111] Tasks: 0 done / 92 total; 50 can run I0803 22:18:07.622049 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:18:07.622078 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0803 22:18:07.622799 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:07.622805 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:18:07.627376 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:18:07.627546 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:07.629897 20401 executor.go:111] Tasks: 50 done / 92 total; 19 can run I0803 22:18:07.635146 20401 executor.go:111] Tasks: 69 done / 92 total; 21 can run I0803 22:18:07.635719 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:07.635978 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:07.637248 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:07.638206 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:07.638326 20401 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0803 22:18:07.638820 20401 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0803 22:18:07.693282 20401 target.go:112] Terraform output is in /tmp/test149217674/out I0803 22:18:07.697325 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:07.697566 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:07.697624 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:07.697678 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:07.697731 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:07.697777 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:07.697808 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:07.697846 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:07.697876 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:07.722763 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:07.725669 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:07.725773 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:07.764895 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0803 22:18:07.863748 20401 executor.go:111] Tasks: 0 done / 92 total; 50 can run W0803 22:18:07.865669 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:18:07.866352 20401 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0803 22:18:07.866402 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:07.866507 20401 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0803 22:18:07.866526 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:18:07.866792 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:18:07.867027 20401 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0803 22:18:07.867060 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:07.867208 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0803 22:18:07.871552 20401 executor.go:111] Tasks: 50 done / 92 total; 19 can run I0803 22:18:07.879421 20401 executor.go:111] Tasks: 69 done / 92 total; 21 can run I0803 22:18:07.879841 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:07.880056 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:07.881171 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:07.882452 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:07.882594 20401 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0803 22:18:07.882870 20401 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0803 22:18:07.886042 20401 target.go:168] Cloudformation output is in /tmp/test2436452170/out --- PASS: TestExternalLoadBalancer (0.44s) === RUN TestPhaseIAM integration_test.go:737: unable to test w/o allowing failed validation --- SKIP: TestPhaseIAM (0.00s) === RUN TestPhaseCluster integration_test.go:745: unable to test w/o allowing failed validation --- SKIP: TestPhaseCluster (0.00s) === RUN TestMixedInstancesASG I0803 22:18:07.896106 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:07.896337 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:07.896387 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:07.896431 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:07.896479 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:07.896560 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:07.896593 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:07.896634 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:07.896663 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:07.934192 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:07.937371 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:07.937468 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:07.937531 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:07.937587 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:07.937639 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:07.937687 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:07.937744 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:07.976277 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:08.080308 20401 executor.go:111] Tasks: 0 done / 103 total; 49 can run I0803 22:18:08.080733 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:18:08.080799 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:08.080915 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:18:08.080949 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:08.081051 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0803 22:18:08.081176 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:18:08.081230 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:18:08.083535 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:08.088437 20401 executor.go:111] Tasks: 49 done / 103 total; 23 can run I0803 22:18:08.094067 20401 executor.go:111] Tasks: 72 done / 103 total; 27 can run I0803 22:18:08.094291 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.094514 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.094741 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.095054 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.097876 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.101204 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.101484 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.101797 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.101941 20401 executor.go:111] Tasks: 99 done / 103 total; 4 can run I0803 22:18:08.102402 20401 executor.go:111] Tasks: 103 done / 103 total; 0 can run I0803 22:18:08.179786 20401 target.go:112] Terraform output is in /tmp/test173631697/out I0803 22:18:08.184478 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:08.184703 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:08.184762 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:08.184817 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:08.184870 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:08.184915 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:08.184944 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:08.184981 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:08.185011 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:08.217527 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:08.220498 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.220589 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.220647 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.220708 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.220762 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.220810 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.220856 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.260352 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0803 22:18:08.360735 20401 executor.go:111] Tasks: 0 done / 103 total; 49 can run W0803 22:18:08.361568 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0803 22:18:08.361581 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:18:08.361636 20401 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0803 22:18:08.361670 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:08.361768 20401 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0803 22:18:08.361815 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:18:08.361910 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0803 22:18:08.361919 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0803 22:18:08.361989 20401 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0803 22:18:08.362016 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:08.362149 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "mixedinstances.example.com" },{ Key: "kubernetes.io/cluster/mixedinstances.example.com", Value: "owned" },{ Key: "Name", Value: "mixedinstances.example.com" }] }] } I0803 22:18:08.366201 20401 executor.go:111] Tasks: 49 done / 103 total; 23 can run I0803 22:18:08.376514 20401 executor.go:111] Tasks: 72 done / 103 total; 27 can run I0803 22:18:08.376929 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.377092 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.377456 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.377819 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.379549 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.379784 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.380049 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.380327 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.380458 20401 executor.go:111] Tasks: 99 done / 103 total; 4 can run I0803 22:18:08.380980 20401 executor.go:111] Tasks: 103 done / 103 total; 0 can run I0803 22:18:08.388801 20401 target.go:168] Cloudformation output is in /tmp/test3354169359/out --- PASS: TestMixedInstancesASG (0.51s) === RUN TestMixedInstancesSpotASG I0803 22:18:08.405162 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:08.405394 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:08.405464 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:08.405522 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:08.405591 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:08.405687 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:08.405718 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:08.405755 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:08.405785 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:08.436909 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:08.439489 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.439565 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.439614 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.439665 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.439712 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.439753 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.439791 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.478605 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:08.566754 20401 executor.go:111] Tasks: 0 done / 103 total; 49 can run I0803 22:18:08.567269 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:08.567330 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0803 22:18:08.567375 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:08.567347 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:08.567404 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0803 22:18:08.567435 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:18:08.567445 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:18:08.567563 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:08.569805 20401 executor.go:111] Tasks: 49 done / 103 total; 23 can run I0803 22:18:08.580384 20401 executor.go:111] Tasks: 72 done / 103 total; 27 can run I0803 22:18:08.580754 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.581121 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.581342 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.581570 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.582897 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.583301 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.583722 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.584007 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.584183 20401 executor.go:111] Tasks: 99 done / 103 total; 4 can run I0803 22:18:08.584498 20401 executor.go:111] Tasks: 103 done / 103 total; 0 can run I0803 22:18:08.666501 20401 target.go:112] Terraform output is in /tmp/test2298841046/out I0803 22:18:08.672345 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:08.672574 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:08.672629 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:08.672678 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:08.672730 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:08.672774 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:08.672805 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:08.672845 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:08.672874 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:08.705570 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:08.708380 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.708463 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.708516 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.708569 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.708621 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.708664 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.708704 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.747969 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0803 22:18:08.847774 20401 executor.go:111] Tasks: 0 done / 103 total; 49 can run I0803 22:18:08.848129 20401 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0803 22:18:08.848159 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0803 22:18:08.848243 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:18:08.848249 20401 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0803 22:18:08.848651 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:18:08.848798 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0803 22:18:08.848801 20401 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key W0803 22:18:08.848897 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:18:08.848835 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:18:08.848961 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:18:08.848943 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "mixedinstances.example.com" },{ Key: "KubernetesCluster", Value: "mixedinstances.example.com" },{ Key: "kubernetes.io/cluster/mixedinstances.example.com", Value: "owned" }] }] } I0803 22:18:08.852255 20401 executor.go:111] Tasks: 49 done / 103 total; 23 can run I0803 22:18:08.861137 20401 executor.go:111] Tasks: 72 done / 103 total; 27 can run I0803 22:18:08.861411 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.861581 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.861765 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.861993 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.862614 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.864474 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.864602 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.864870 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:08.865020 20401 executor.go:111] Tasks: 99 done / 103 total; 4 can run I0803 22:18:08.865403 20401 executor.go:111] Tasks: 103 done / 103 total; 0 can run I0803 22:18:08.871489 20401 target.go:168] Cloudformation output is in /tmp/test2502515765/out --- PASS: TestMixedInstancesSpotASG (0.49s) === RUN TestContainerd I0803 22:18:08.890428 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:08.890692 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:08.890743 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:08.890844 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:08.890898 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:08.890942 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:08.890973 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:08.891011 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:08.891040 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:08.913566 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:08.915842 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:08.915918 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:08.954677 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0803 22:18:09.056270 20401 executor.go:111] Tasks: 0 done / 85 total; 43 can run W0803 22:18:09.057133 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:18:09.058274 20401 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0803 22:18:09.058317 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:09.058422 20401 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0803 22:18:09.059268 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:18:09.059608 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:18:09.059830 20401 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0803 22:18:09.059859 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:09.060179 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "containerd.example.com" },{ Key: "KubernetesCluster", Value: "containerd.example.com" },{ Key: "kubernetes.io/cluster/containerd.example.com", Value: "owned" }] }] } I0803 22:18:09.064027 20401 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0803 22:18:09.069617 20401 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0803 22:18:09.069922 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:09.070302 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:09.071031 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:09.072622 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:09.072766 20401 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0803 22:18:09.073043 20401 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0803 22:18:09.076285 20401 target.go:168] Cloudformation output is in /tmp/test2241767191/out --- PASS: TestContainerd (0.20s) === RUN TestContainerdCustom I0803 22:18:09.086133 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:09.086367 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:09.086419 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:09.086510 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:09.086562 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:09.086607 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:09.086636 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:09.086676 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:09.086705 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:09.116331 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:09.119439 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:09.119539 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:09.159136 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0803 22:18:09.260797 20401 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0803 22:18:09.261258 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:18:09.261389 20401 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key W0803 22:18:09.261399 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:18:09.261414 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:09.261514 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "containerd.example.com" },{ Key: "KubernetesCluster", Value: "containerd.example.com" },{ Key: "kubernetes.io/cluster/containerd.example.com", Value: "owned" }] }] } W0803 22:18:09.261947 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:18:09.264191 20401 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0803 22:18:09.264247 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:09.264386 20401 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0803 22:18:09.268097 20401 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0803 22:18:09.274082 20401 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0803 22:18:09.274435 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:09.274691 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:09.275739 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:09.277213 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:09.277357 20401 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0803 22:18:09.277612 20401 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0803 22:18:09.280681 20401 target.go:168] Cloudformation output is in /tmp/test3408818018/out --- PASS: TestContainerdCustom (0.20s) === RUN TestDockerCustom I0803 22:18:09.290465 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:09.290699 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:09.290753 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:09.290846 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:09.290902 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:09.290945 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:09.290976 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:09.291016 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:09.291045 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:09.320085 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:09.323264 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:09.323366 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:09.363971 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0803 22:18:09.466483 20401 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0803 22:18:09.467058 20401 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0803 22:18:09.467100 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:09.467278 20401 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0803 22:18:09.467422 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:18:09.467535 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0803 22:18:09.468724 20401 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0803 22:18:09.468793 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:09.468922 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "docker.example.com" },{ Key: "KubernetesCluster", Value: "docker.example.com" },{ Key: "kubernetes.io/cluster/docker.example.com", Value: "owned" }] }] } W0803 22:18:09.467562 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:18:09.473987 20401 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0803 22:18:09.479864 20401 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0803 22:18:09.480090 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:09.480486 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:09.481511 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:09.483152 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:09.483298 20401 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0803 22:18:09.483560 20401 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0803 22:18:09.486583 20401 target.go:168] Cloudformation output is in /tmp/test472252108/out --- PASS: TestDockerCustom (0.21s) === RUN TestAPIServerNodes I0803 22:18:09.496640 20401 featureflag.go:162] FeatureFlag "APIServerNodes"=true I0803 22:18:09.496682 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:09.496907 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:09.496958 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:09.497042 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:09.497094 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:09.497138 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:09.497167 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:09.497203 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:09.497231 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:09.527638 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.10 ********************************************************************************* I0803 22:18:09.530595 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:09.530695 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:09.530759 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:09.572292 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0803 22:18:09.730053 20401 executor.go:111] Tasks: 0 done / 97 total; 47 can run I0803 22:18:09.730335 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:18:09.730481 20401 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0803 22:18:09.730499 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:09.730599 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } W0803 22:18:09.730781 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:18:09.730842 20401 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0803 22:18:09.730871 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0803 22:18:09.730974 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:18:09.730967 20401 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0803 22:18:09.733799 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:18:09.737784 20401 executor.go:111] Tasks: 47 done / 97 total; 24 can run I0803 22:18:09.743274 20401 executor.go:111] Tasks: 71 done / 97 total; 23 can run I0803 22:18:09.743551 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:09.743910 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:09.744092 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:09.745147 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:09.745518 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:09.746913 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:09.747061 20401 executor.go:111] Tasks: 94 done / 97 total; 3 can run I0803 22:18:09.747366 20401 executor.go:111] Tasks: 97 done / 97 total; 0 can run I0803 22:18:09.751467 20401 target.go:168] Cloudformation output is in /tmp/test4065995392/out I0803 22:18:09.766652 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:09.766913 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:09.766970 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:09.767021 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:09.767074 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:09.767119 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:09.767149 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:09.767188 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:09.767218 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:09.794113 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.10 ********************************************************************************* I0803 22:18:09.797158 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:09.797263 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:09.797329 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:09.835802 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:09.995392 20401 executor.go:111] Tasks: 0 done / 97 total; 47 can run W0803 22:18:09.995938 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:09.996495 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0803 22:18:09.996600 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:09.996540 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:09.996679 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0803 22:18:09.996788 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:09.996749 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:10.002625 20401 executor.go:111] Tasks: 47 done / 97 total; 24 can run I0803 22:18:10.008370 20401 executor.go:111] Tasks: 71 done / 97 total; 23 can run I0803 22:18:10.008738 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:10.009022 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:10.009310 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:10.010032 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:10.010537 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:10.011827 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:10.012052 20401 executor.go:111] Tasks: 94 done / 97 total; 3 can run I0803 22:18:10.012587 20401 executor.go:111] Tasks: 97 done / 97 total; 0 can run I0803 22:18:10.074919 20401 target.go:112] Terraform output is in /tmp/test1318874993/out I0803 22:18:10.079563 20401 featureflag.go:162] FeatureFlag "APIServerNodes"=false --- PASS: TestAPIServerNodes (0.58s) === RUN TestNTHQueueProcessor I0803 22:18:10.079698 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:10.079948 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:10.080004 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:10.080057 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:10.080110 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:10.080218 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:10.080248 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:10.080287 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:10.080317 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:10.109760 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0803 22:18:10.112809 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:10.112908 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:11.479815 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:11.602869 20401 executor.go:111] Tasks: 0 done / 99 total; 45 can run W0803 22:18:11.603173 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:18:11.603472 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:11.603495 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:18:11.603545 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:11.603542 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:11.603673 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:18:11.606292 20401 executor.go:111] Tasks: 45 done / 99 total; 24 can run I0803 22:18:11.615522 20401 executor.go:111] Tasks: 69 done / 99 total; 26 can run I0803 22:18:11.616074 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:11.616346 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:11.617246 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:11.618575 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:11.618694 20401 executor.go:111] Tasks: 95 done / 99 total; 2 can run I0803 22:18:11.619055 20401 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0803 22:18:11.619283 20401 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0803 22:18:11.681740 20401 target.go:112] Terraform output is in /tmp/test2751794325/out I0803 22:18:11.686780 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:11.687008 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:11.687068 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:11.687126 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:11.687182 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:11.687231 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:11.687261 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:11.687300 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:11.687329 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:11.711860 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0803 22:18:11.714700 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:11.714791 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:11.755998 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0803 22:18:11.879684 20401 executor.go:111] Tasks: 0 done / 99 total; 45 can run W0803 22:18:11.880475 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0803 22:18:11.881847 20401 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0803 22:18:11.882699 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:11.882834 20401 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0803 22:18:11.882888 20401 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0803 22:18:11.882869 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:11.882993 20401 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0803 22:18:11.882909 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:11.883032 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" }] }] } I0803 22:18:11.887861 20401 executor.go:111] Tasks: 45 done / 99 total; 24 can run I0803 22:18:11.893516 20401 executor.go:111] Tasks: 69 done / 99 total; 26 can run I0803 22:18:11.894240 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:11.894515 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:11.895291 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:11.896763 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:11.896897 20401 executor.go:111] Tasks: 95 done / 99 total; 2 can run I0803 22:18:11.897165 20401 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0803 22:18:11.897350 20401 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0803 22:18:11.901070 20401 target.go:168] Cloudformation output is in /tmp/test2974368379/out --- PASS: TestNTHQueueProcessor (1.84s) === RUN TestCustomIRSA I0803 22:18:11.915730 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:11.916134 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:11.916186 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:11.916234 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:11.916284 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:11.916370 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:11.916400 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:11.916436 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:11.916466 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:11.945119 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0803 22:18:11.948373 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:11.948465 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:11.987266 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:12.542756 20401 executor.go:111] Tasks: 0 done / 98 total; 50 can run W0803 22:18:12.543034 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target W0803 22:18:12.543254 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:12.543888 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:12.543952 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:18:12.544962 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:12.545277 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:18:12.551791 20401 executor.go:111] Tasks: 50 done / 98 total; 25 can run I0803 22:18:12.557015 20401 executor.go:111] Tasks: 75 done / 98 total; 21 can run I0803 22:18:12.557461 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:12.557626 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:12.558599 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:12.559908 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:12.560034 20401 executor.go:111] Tasks: 96 done / 98 total; 2 can run I0803 22:18:12.560371 20401 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0803 22:18:12.616460 20401 target.go:112] Terraform output is in /tmp/test2484416080/out --- PASS: TestCustomIRSA (0.71s) === RUN TestCustomIRSA119 I0803 22:18:12.627619 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:12.628315 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:12.628398 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:12.628480 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:12.628566 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:12.628673 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:12.628733 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:12.628819 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:12.628869 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:12.657017 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.19.16 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.19.16 ********************************************************************************* I0803 22:18:12.660624 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:12.660744 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:12.701042 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:12.800165 20401 executor.go:111] Tasks: 0 done / 96 total; 48 can run W0803 22:18:12.800503 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:12.800970 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0803 22:18:12.801223 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:12.801390 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0803 22:18:12.801434 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:12.801536 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:18:12.803297 20401 executor.go:111] Tasks: 48 done / 96 total; 25 can run I0803 22:18:12.808551 20401 executor.go:111] Tasks: 73 done / 96 total; 21 can run I0803 22:18:12.808912 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:12.809187 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:12.810352 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:12.811311 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:12.811438 20401 executor.go:111] Tasks: 94 done / 96 total; 2 can run I0803 22:18:12.811754 20401 executor.go:111] Tasks: 96 done / 96 total; 0 can run I0803 22:18:12.867014 20401 target.go:112] Terraform output is in /tmp/test1559335026/out --- PASS: TestCustomIRSA119 (0.24s) === RUN TestClusterNameDigit I0803 22:18:12.871635 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:12.871882 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:12.871940 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:12.871996 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:12.872049 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:12.872166 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:12.872199 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:12.872238 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:12.872267 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:12.902500 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0803 22:18:12.906340 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:12.906446 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:12.945475 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:13.050385 20401 executor.go:111] Tasks: 0 done / 93 total; 47 can run W0803 22:18:13.051073 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:13.051093 20401 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0803 22:18:13.051151 20401 warmpool.go:124] ASG warm pool is not supported by the terraform target I0803 22:18:13.051146 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:13.051257 20401 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0803 22:18:13.051390 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.123.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:13.053092 20401 executor.go:111] Tasks: 47 done / 93 total; 23 can run I0803 22:18:13.058200 20401 executor.go:111] Tasks: 70 done / 93 total; 21 can run I0803 22:18:13.058594 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:13.058870 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:13.059679 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:13.061014 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:13.061132 20401 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0803 22:18:13.061477 20401 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0803 22:18:13.113957 20401 target.go:112] Terraform output is in /tmp/test1249736148/out --- PASS: TestClusterNameDigit (0.25s) === RUN TestLifecycleMinimalAWS I0803 22:18:13.118498 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:13.118758 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:13.118845 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:13.118902 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:13.118958 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:13.119005 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:13.119035 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:13.119075 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:13.119102 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:13.119167 20401 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal.example.com I0803 22:18:13.146389 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:13.149343 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:13.149451 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0803 22:18:13.190401 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:13.286339 20401 iamrole.go:89] ListRoles: { } I0803 22:18:13.288842 20401 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0803 22:18:13.289660 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:13.289614 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0803 22:18:13.289988 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I0803 22:18:13.290159 20401 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal.example.com" }] } I0803 22:18:13.290192 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0803 22:18:13.290404 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } I0803 22:18:13.290473 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["minimal.example.com"] }] } I0803 22:18:13.290644 20401 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0803 22:18:13.290934 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0803 22:18:13.291195 20401 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0803 22:18:13.293199 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0803 22:18:13.293276 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:13.293257 20401 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0803 22:18:13.293359 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:13.293464 20401 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0803 22:18:13.293545 20401 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0803 22:18:13.293786 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0803 22:18:13.294044 20401 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0803 22:18:13.294236 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0803 22:18:13.294319 20401 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0803 22:18:13.294385 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0803 22:18:13.294451 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:13.294512 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0803 22:18:13.295075 20401 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0803 22:18:13.295299 20401 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0803 22:18:13.295241 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0803 22:18:13.295384 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0803 22:18:13.295481 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0803 22:18:13.295665 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0803 22:18:13.295706 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0803 22:18:13.295847 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0803 22:18:13.295877 20401 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0803 22:18:13.295827 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0803 22:18:13.296164 20401 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0803 22:18:13.296174 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0803 22:18:13.296411 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:13.296602 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0803 22:18:13.296880 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:13.296847 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0803 22:18:13.297042 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0803 22:18:13.297098 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:18:13.297144 20401 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0803 22:18:13.297196 20401 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0803 22:18:13.297300 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0803 22:18:13.297393 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0803 22:18:13.297477 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0803 22:18:13.297544 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:13.297591 20401 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0803 22:18:13.297699 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0803 22:18:13.297759 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0803 22:18:13.297812 20401 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0803 22:18:13.297900 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0803 22:18:13.297957 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:13.297761 20401 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0803 22:18:13.300483 20401 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0803 22:18:13.300704 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:13.300891 20401 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0803 22:18:13.300960 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:13.301113 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:13.301179 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:13.301337 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:13.301454 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0803 22:18:13.301659 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:13.301839 20401 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0803 22:18:13.301927 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0803 22:18:13.302130 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:13.302296 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-1to2379-masters.minimal.example.com" }] }] } I0803 22:18:13.302627 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0803 22:18:13.302977 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-4003to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0803 22:18:13.303289 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0803 22:18:13.303441 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0803 22:18:13.303577 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0803 22:18:13.303705 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0803 22:18:13.303832 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-udp-1to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0803 22:18:13.303968 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0803 22:18:13.304128 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0803 22:18:13.304274 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0803 22:18:13.304433 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-2382to4000-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0803 22:18:13.304583 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0803 22:18:13.304689 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0803 22:18:13.304790 20401 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0803 22:18:13.305154 20401 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0803 22:18:13.305207 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:13.305245 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:13.305312 20401 dns.go:238] Pre-creating DNS records I0803 22:18:13.305342 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:13.312329 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:13.317862 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:13.317966 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0803 22:18:13.357765 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:13.460214 20401 iamrole.go:89] ListRoles: { } I0803 22:18:13.461846 20401 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0803 22:18:13.462060 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0803 22:18:13.462250 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0803 22:18:13.462314 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0803 22:18:13.462404 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:13.462517 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0803 22:18:13.462794 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:13.462855 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0803 22:18:13.463051 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:13.463123 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:13.463260 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0803 22:18:13.464914 20401 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0803 22:18:13.465046 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:13.465162 20401 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0803 22:18:13.465188 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:13.465302 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0803 22:18:13.465310 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0803 22:18:13.465514 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:13.465565 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] }] } I0803 22:18:13.465758 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:18:13.465807 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0803 22:18:13.465943 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0803 22:18:13.466097 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:13.466157 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["minimal.example.com"] }] } I0803 22:18:13.474107 20401 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0803 22:18:13.474298 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:13.474544 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:13.474715 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:13.474914 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:13.475066 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:13.475338 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:13.475501 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:13.477172 20401 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0803 22:18:13.477373 20401 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0803 22:18:13.480229 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0803 22:18:13.480339 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0803 22:18:13.480416 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0803 22:18:13.480478 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0803 22:18:13.480553 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0803 22:18:13.480622 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0803 22:18:13.480683 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0803 22:18:13.480748 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0803 22:18:13.480810 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0803 22:18:13.480876 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0803 22:18:13.480939 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0803 22:18:13.480984 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0803 22:18:13.481016 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0803 22:18:13.481051 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0803 22:18:13.481090 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0803 22:18:13.481124 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0803 22:18:13.481161 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0803 22:18:13.481199 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0803 22:18:13.481238 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0803 22:18:13.481275 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0803 22:18:13.481312 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0803 22:18:13.481349 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0803 22:18:13.481389 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0803 22:18:13.481429 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0803 22:18:13.481465 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0803 22:18:13.482759 20401 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0803 22:18:13.482793 20401 keypairs.go:95] DescribeKeyPairs: { } I0803 22:18:13.482837 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0803 22:18:13.482892 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0803 22:18:13.482951 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0803 22:18:13.483014 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0803 22:18:13.483065 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0803 22:18:13.483126 20401 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0803 22:18:13.483200 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0803 22:18:13.483266 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0803 22:18:13.483355 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0803 22:18:13.483449 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0803 22:18:13.483520 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } W0803 22:18:13.483607 20401 tags.go:48] unknown cluster tag on vpc:vpc-12345678: "kubernetes.io/cluster/minimal.example.com"="" W0803 22:18:13.483668 20401 api.go:53] PageSize not implemented I0803 22:18:13.483710 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0803 22:18:13.483769 20401 loadbalancers.go:34] PageSize not implemented I0803 22:18:13.483808 20401 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0803 22:18:13.483861 20401 targetgroups.go:35] PageSize not implemented I0803 22:18:13.483897 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0803 22:18:13.483999 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:13.484056 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:13.484145 20401 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0803 22:18:13.484195 20401 iamrole.go:89] ListRoles: { } I0803 22:18:13.484265 20401 internetgateways.go:101] DescribeInternetGateways: { } I0803 22:18:13.484313 20401 routetable.go:53] DescribeRouteTables: { } I0803 22:18:13.484347 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:master-us-test-1a.masters.minimal.example.com ok I0803 22:18:13.484928 20401 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } autoscaling-group:nodes.minimal.example.com ok I0803 22:18:13.485001 20401 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0803 22:18:13.485031 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } volume:vol-1 ok I0803 22:18:13.485084 20401 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } iam-instance-profile:nodes.minimal.example.com ok I0803 22:18:13.485133 20401 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0803 22:18:13.485001 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.minimal.example.com. ok I0803 22:18:13.485190 20401 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } iam-instance-profile:masters.minimal.example.com ok I0803 22:18:13.485097 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0803 22:18:13.485263 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:13.485285 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-2 ok I0803 22:18:13.485372 20401 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0803 22:18:13.485445 20401 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0803 22:18:13.485511 20401 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0803 22:18:13.485573 20401 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-1 ok I0803 22:18:13.485644 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:13.485694 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0803 22:18:13.485897 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:13.485911 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0803 22:18:13.485989 20401 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal.example.com" } I0803 22:18:13.486046 20401 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal.example.com I0803 22:18:13.486062 20401 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0803 22:18:13.486107 20401 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal.example.com" } iam-role:nodes.minimal.example.com ok I0803 22:18:13.486060 20401 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0803 22:18:13.486155 20401 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal.example.com" } I0803 22:18:13.486289 20401 iamrole.go:180] ListAttachedRolePolicies: masters.minimal.example.com I0803 22:18:13.486321 20401 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0803 22:18:13.486381 20401 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal.example.com" } iam-role:masters.minimal.example.com ok I0803 22:18:13.486511 20401 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0803 22:18:13.486677 20401 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0803 22:18:13.486819 20401 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0803 22:18:13.487240 20401 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleMinimalAWS (0.37s) === RUN TestLifecycleMinimalOpenstack I0803 22:18:13.487512 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster minimal-openstack.k8s.local I0803 22:18:13.496905 20401 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:13.563205 20401 apply_cluster.go:467] Gossip DNS: skipping DNS validation W0803 22:18:13.717022 20401 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0803 22:18:13.717081 20401 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0803 22:18:13.733100 20401 executor.go:111] Tasks: 0 done / 96 total; 42 can run I0803 22:18:13.742947 20401 executor.go:111] Tasks: 42 done / 96 total; 45 can run I0803 22:18:13.790375 20401 executor.go:111] Tasks: 87 done / 96 total; 5 can run I0803 22:18:13.793678 20401 executor.go:111] Tasks: 92 done / 96 total; 4 can run W0803 22:18:13.796755 20401 bootstrapscript.go:166] exporting username and password. Consider using application credentials instead. W0803 22:18:13.796764 20401 bootstrapscript.go:166] exporting username and password. Consider using application credentials instead. W0803 22:18:13.796972 20401 bootstrapscript.go:166] exporting username and password. Consider using application credentials instead. I0803 22:18:13.802288 20401 executor.go:111] Tasks: 96 done / 96 total; 0 can run ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:13.849695 20401 apply_cluster.go:467] Gossip DNS: skipping DNS validation W0803 22:18:13.992236 20401 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0803 22:18:13.992300 20401 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0803 22:18:14.011531 20401 executor.go:111] Tasks: 0 done / 96 total; 42 can run I0803 22:18:14.014780 20401 executor.go:111] Tasks: 42 done / 96 total; 45 can run I0803 22:18:14.027682 20401 executor.go:111] Tasks: 87 done / 96 total; 5 can run I0803 22:18:14.030596 20401 executor.go:111] Tasks: 92 done / 96 total; 4 can run W0803 22:18:14.033979 20401 bootstrapscript.go:166] exporting username and password. Consider using application credentials instead. W0803 22:18:14.034438 20401 bootstrapscript.go:166] exporting username and password. Consider using application credentials instead. I0803 22:18:14.036514 20401 executor.go:111] Tasks: 96 done / 96 total; 0 can run Instance:597407c3-776f-48ba-aa4c-3ccbe97b1cf7 ok Instance:ba00f3d1-c446-4351-956b-69e30991ed2a ok Instance:4525f4c0-8765-4ed0-9c86-d7eb2a6c5072 ok Network:888c54ac-3147-4244-93d8-cae059bd70ab ok SecurityGroup:92414643-968f-471b-be77-6acf2704f406 ok Subnet:6dd1b449-340d-4f9b-9c1a-64ec1ff737f4 ok Port:7c6f6913-ead4-47ab-95c9-107d7fc0f6c0 ok SecurityGroup:01d5035a-28e6-4ad5-b9c4-424caaac8fa9 ok Port:ec2d7502-8cc1-4d25-ac1d-ab5d38898171 ok SSHKey:kubernetes-minimal-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok SecurityGroup:590436a3-c0d6-4952-8ef5-74a4c19a507c ok Port:f8da8495-b746-4a01-98d6-8643bce75d3d ok ServerGroup:baedee52-774a-4a1d-b0af-fd9dc577c8c2 ok ServerGroup:ba7ee8c4-3e57-45ce-8742-87820a6fa2ff ok --- PASS: TestLifecycleMinimalOpenstack (1.65s) === RUN TestLifecycleMinimalGCE I0803 22:18:15.139998 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:486: running lifecycle test for cluster minimal-gce.example.com I0803 22:18:15.141889 20401 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* W0803 22:18:15.383688 20401 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0803 22:18:15.383789 20401 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0803 22:18:15.383905 20401 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0803 22:18:15.383944 20401 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0803 22:18:15.383971 20401 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0803 22:18:15.384005 20401 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0803 22:18:15.384079 20401 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0803 22:18:15.385849 20401 executor.go:111] Tasks: 0 done / 64 total; 41 can run W0803 22:18:15.389172 20401 executor.go:139] error running task "ProjectIAMBinding/serviceaccount-control-plane" (9s remaining to succeed): error updating IAM for project testproject: googleapi: Error 404: etag I0803 22:18:15.389252 20401 executor.go:111] Tasks: 40 done / 64 total; 18 can run I0803 22:18:15.394641 20401 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0803 22:18:15.397353 20401 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0803 22:18:15.397601 20401 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0803 22:18:15.397700 20401 dns.go:238] Pre-creating DNS records ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* W0803 22:18:15.611490 20401 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0803 22:18:15.611573 20401 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0803 22:18:15.611667 20401 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0803 22:18:15.611701 20401 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0803 22:18:15.611726 20401 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0803 22:18:15.611755 20401 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0803 22:18:15.611825 20401 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0803 22:18:15.613353 20401 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0803 22:18:15.621602 20401 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0803 22:18:15.627058 20401 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0803 22:18:15.630287 20401 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0803 22:18:15.630449 20401 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0803 22:18:15.632417 20401 gce.go:98] Scanning zones: [us-test1-a] I0803 22:18:15.632823 20401 gce.go:1070] backend services not found, assuming none exist in project: "testproject" region: "us-test1" I0803 22:18:15.632848 20401 gce.go:1070] backend services not found, assuming none exist in project: "testproject" region: "us-test1" ServiceAccount:projects/testproject/serviceAccounts/node-minimal-gce-example-com@testproject.iam.gserviceaccount.com ok FirewallRule:master-to-master-minimal-gce-example-com ok FirewallRule:node-to-master-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-nodes-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-ipv6-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-minimal-gce-example-com ok FirewallRule:node-to-node-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-master-us-test1-a-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-ipv6-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-minimal-gce-example-com ok Disk:d1-etcd-events-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-ipv6-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/control-plane-minimal-g-fu1mg6@testproject.iam.gserviceaccount.com ok Disk:d1-etcd-main-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-ipv6-minimal-gce-example-com ok FirewallRule:master-to-node-minimal-gce-example-com ok InstanceTemplate:nodes-minimal-gce-example-com-1659565095 ok InstanceTemplate:master-us-test1-a-minimal-do16cp-1659565095 ok Subnet:us-test1-minimal-gce-example-com ok Network:minimal-gce-example-com ok --- PASS: TestLifecycleMinimalGCE (0.49s) === RUN TestLifecycleFloatingIPOpenstack I0803 22:18:15.634363 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster floatingip-openstack.k8s.local I0803 22:18:15.642403 20401 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:15.712345 20401 apply_cluster.go:467] Gossip DNS: skipping DNS validation W0803 22:18:15.857976 20401 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0803 22:18:15.858016 20401 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0803 22:18:15.873814 20401 executor.go:111] Tasks: 0 done / 98 total; 43 can run I0803 22:18:15.878738 20401 executor.go:111] Tasks: 43 done / 98 total; 45 can run I0803 22:18:15.950810 20401 executor.go:111] Tasks: 88 done / 98 total; 6 can run I0803 22:18:15.953644 20401 executor.go:111] Tasks: 94 done / 98 total; 4 can run W0803 22:18:15.955974 20401 bootstrapscript.go:166] exporting username and password. Consider using application credentials instead. W0803 22:18:15.956321 20401 bootstrapscript.go:166] exporting username and password. Consider using application credentials instead. W0803 22:18:15.956414 20401 bootstrapscript.go:166] exporting username and password. Consider using application credentials instead. I0803 22:18:15.965934 20401 executor.go:111] Tasks: 98 done / 98 total; 0 can run ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:16.015235 20401 apply_cluster.go:467] Gossip DNS: skipping DNS validation W0803 22:18:16.164104 20401 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0803 22:18:16.164159 20401 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0803 22:18:16.188946 20401 executor.go:111] Tasks: 0 done / 98 total; 43 can run I0803 22:18:16.192692 20401 executor.go:111] Tasks: 43 done / 98 total; 45 can run I0803 22:18:16.203943 20401 executor.go:111] Tasks: 88 done / 98 total; 6 can run I0803 22:18:16.206343 20401 executor.go:111] Tasks: 94 done / 98 total; 4 can run W0803 22:18:16.209488 20401 bootstrapscript.go:166] exporting username and password. Consider using application credentials instead. W0803 22:18:16.209937 20401 bootstrapscript.go:166] exporting username and password. Consider using application credentials instead. W0803 22:18:16.210264 20401 bootstrapscript.go:166] exporting username and password. Consider using application credentials instead. I0803 22:18:16.211608 20401 executor.go:111] Tasks: 98 done / 98 total; 0 can run Router-IF:50ea03cc-ff2a-4c53-8cef-a9d6c7740822 ok Port:1e6f097d-a098-4309-856f-2f869446aa8e ok SecurityGroup:9834c858-2ab2-47f0-8b7f-37bc860a2d81 ok Instance:099d5141-80b0-48a8-aa3e-4c27426056f6 ok Network:c31624b5-3b0c-4e37-9585-0de3248d958d ok Port:b946393a-8dd7-4b85-ac97-bb9e34475d9c ok SecurityGroup:2d23fd3c-862d-44a6-bf90-770b0c8e7efb ok Instance:9076166e-1891-4cb5-b4ce-12c8877342b6 ok SSHKey:kubernetes-floatingip-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok Router:a4897c22-adea-45a3-8246-030d43a5eea7 ok ServerGroup:c40eb4db-3340-40fe-b1ec-a231fec52148 ok ServerGroup:4c527304-2d39-44b1-ac09-604fd88eaa0e ok Port:4c7581cb-454e-4b06-a9fe-945b2ba84a55 ok Port:c45bd0f8-8f96-40f5-b0a7-dda2c4296beb ok Subnet:50ea03cc-ff2a-4c53-8cef-a9d6c7740822 ok SecurityGroup:635f1c01-d5bf-49af-afd9-6cc0759b2907 ok Instance:bcfdd992-d453-4970-85f7-75c32654c6d9 ok --- PASS: TestLifecycleFloatingIPOpenstack (1.69s) === RUN TestLifecyclePrivateCalico I0803 22:18:17.327688 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:17.327966 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:17.328021 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:17.328071 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:17.328125 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:17.328172 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:17.328202 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:17.328239 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:17.328270 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:17.328331 20401 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatecalico.example.com I0803 22:18:17.357084 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0803 22:18:17.360492 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:17.360588 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:17.360659 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:17.401598 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:17.824509 20401 iamrole.go:89] ListRoles: { } I0803 22:18:17.826901 20401 executor.go:111] Tasks: 0 done / 124 total; 50 can run I0803 22:18:17.827486 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:17.827489 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } I0803 22:18:17.827655 20401 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.privatecalico.example.com" }] } I0803 22:18:17.827570 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0803 22:18:17.827943 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:17.827929 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } I0803 22:18:17.828030 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:17.828081 20401 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0803 22:18:17.828139 20401 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0803 22:18:17.828324 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0803 22:18:17.828316 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } I0803 22:18:17.828392 20401 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", Tags: [{ Key: "Name", Value: "bastions.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0803 22:18:17.828394 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0803 22:18:17.829647 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0803 22:18:17.829662 20401 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0803 22:18:17.829953 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0803 22:18:17.830291 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0803 22:18:17.830696 20401 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0803 22:18:17.830939 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0803 22:18:17.831063 20401 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0803 22:18:17.831134 20401 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0803 22:18:17.831189 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0803 22:18:17.831264 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:17.831307 20401 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0803 22:18:17.831927 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0803 22:18:17.834925 20401 executor.go:111] Tasks: 50 done / 124 total; 28 can run I0803 22:18:17.835063 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:17.835218 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0803 22:18:17.835391 20401 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I0803 22:18:17.835380 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0803 22:18:17.835480 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0803 22:18:17.835600 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I0803 22:18:17.835638 20401 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I0803 22:18:17.835569 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0803 22:18:17.835707 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0803 22:18:17.835733 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0803 22:18:17.835817 20401 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0803 22:18:17.835844 20401 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0803 22:18:17.835832 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0803 22:18:17.835938 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0803 22:18:17.835962 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0803 22:18:17.836126 20401 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0803 22:18:17.836152 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0803 22:18:17.836343 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0803 22:18:17.836346 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.privatecalico.example.com" } ] }], VpcId: "vpc-1" } I0803 22:18:17.836459 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0803 22:18:17.836520 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:17.836582 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0803 22:18:17.836634 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:18:17.836685 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0803 22:18:17.836658 20401 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0803 22:18:17.836800 20401 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0803 22:18:17.836844 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0803 22:18:17.836957 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0803 22:18:17.837066 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I0803 22:18:17.837167 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0803 22:18:17.837266 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:17.837340 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0803 22:18:17.837362 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], VpcId: "vpc-1" } I0803 22:18:17.837437 20401 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0803 22:18:17.837526 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0803 22:18:17.837576 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0803 22:18:17.837654 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0803 22:18:17.837699 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0803 22:18:17.837734 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }], VpcId: "vpc-1" } I0803 22:18:17.837840 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0803 22:18:17.837499 20401 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0803 22:18:17.837905 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0803 22:18:17.837973 20401 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0803 22:18:17.838036 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0803 22:18:17.838097 20401 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] } I0803 22:18:17.838186 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0803 22:18:17.838235 20401 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatecalico.example.com" } ] } I0803 22:18:17.838318 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0803 22:18:17.838411 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:17.838489 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0803 22:18:17.838575 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0803 22:18:17.838621 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0803 22:18:17.838723 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0803 22:18:17.838794 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0803 22:18:17.840688 20401 executor.go:111] Tasks: 78 done / 124 total; 40 can run I0803 22:18:17.840855 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0803 22:18:17.841052 20401 api.go:53] PageSize not implemented I0803 22:18:17.841113 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0803 22:18:17.841190 20401 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0803 22:18:17.841291 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0803 22:18:17.841381 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I0803 22:18:17.841545 20401 tags.go:74] AddTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"], Tags: [{ Key: "Name", Value: "api.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0803 22:18:17.841706 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0803 22:18:17.841666 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } I0803 22:18:17.841796 20401 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0803 22:18:17.841834 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0803 22:18:17.841922 20401 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatecalico-example-0uch4k" } W0803 22:18:17.842016 20401 api.go:53] PageSize not implemented I0803 22:18:17.842048 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0803 22:18:17.842005 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-1to2379-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0803 22:18:17.842285 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0803 22:18:17.842332 20401 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"], Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] } I0803 22:18:17.842335 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0803 22:18:17.842398 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0803 22:18:17.842441 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0803 22:18:17.842535 20401 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0803 22:18:17.842597 20401 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0803 22:18:17.842563 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-4003to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0803 22:18:17.842705 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:17.842791 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:17.842901 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0803 22:18:17.843034 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:17.843111 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:17.843178 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0803 22:18:17.843314 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0803 22:18:17.843420 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:17.843516 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0803 22:18:17.843645 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-udp-1to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0803 22:18:17.843783 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0803 22:18:17.843948 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0803 22:18:17.844082 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-::/0" }] }] } I0803 22:18:17.844224 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0803 22:18:17.844387 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0803 22:18:17.844520 20401 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0803 22:18:17.844571 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0803 22:18:17.844703 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-ingress-tcp-22to22-bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0803 22:18:17.844841 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0803 22:18:17.844975 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0803 22:18:17.845104 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0803 22:18:17.845235 20401 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0803 22:18:17.845286 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-2382to4000-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0803 22:18:17.845424 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:17.845551 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:17.845615 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:17.845792 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:17.845911 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-4-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0803 22:18:17.846119 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0803 22:18:17.846196 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0803 22:18:17.846328 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0803 22:18:17.846534 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0803 22:18:17.846599 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I0803 22:18:17.846757 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:17.846841 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0803 22:18:17.846909 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0803 22:18:17.848111 20401 executor.go:111] Tasks: 118 done / 124 total; 5 can run I0803 22:18:17.848278 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:17.848354 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:17.848459 20401 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-2"] }] } I0803 22:18:17.848459 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:17.848609 20401 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-2", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0803 22:18:17.848714 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0803 22:18:17.848800 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0803 22:18:17.848854 20401 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0803 22:18:17.848924 20401 executor.go:111] Tasks: 123 done / 124 total; 1 can run I0803 22:18:17.848989 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:17.849123 20401 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0803 22:18:17.849192 20401 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0803 22:18:17.849235 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:17.849270 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:17.849322 20401 dns.go:238] Pre-creating DNS records I0803 22:18:17.849349 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:17.855781 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0803 22:18:17.859147 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:17.859245 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:17.859311 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:17.899142 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:18.300844 20401 iamrole.go:89] ListRoles: { } I0803 22:18:18.303256 20401 executor.go:111] Tasks: 0 done / 124 total; 50 can run I0803 22:18:18.303671 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0803 22:18:18.303939 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:18.303930 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0803 22:18:18.304158 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0803 22:18:18.304364 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0803 22:18:18.304483 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0803 22:18:18.304518 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0803 22:18:18.304574 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:18.304627 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:18.304699 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:18.306568 20401 executor.go:111] Tasks: 50 done / 124 total; 28 can run I0803 22:18:18.306801 20401 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I0803 22:18:18.306793 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I0803 22:18:18.307044 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I0803 22:18:18.307052 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:18.307208 20401 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I0803 22:18:18.307168 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0803 22:18:18.307398 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:18.307465 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0803 22:18:18.307603 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0803 22:18:18.307731 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0803 22:18:18.307854 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:18.307945 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0803 22:18:18.308089 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0803 22:18:18.308139 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0803 22:18:18.308247 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0803 22:18:18.308283 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0803 22:18:18.308389 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0803 22:18:18.308491 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0803 22:18:18.308629 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:18:18.308673 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0803 22:18:18.308807 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:18.308885 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0803 22:18:18.313300 20401 executor.go:111] Tasks: 78 done / 124 total; 40 can run I0803 22:18:18.313461 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:18.313625 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0803 22:18:18.313707 20401 api.go:53] PageSize not implemented I0803 22:18:18.313748 20401 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0803 22:18:18.313795 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0803 22:18:18.313897 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0803 22:18:18.313941 20401 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } W0803 22:18:18.314030 20401 api.go:53] PageSize not implemented I0803 22:18:18.314050 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0803 22:18:18.314071 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:18.314127 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0803 22:18:18.314262 20401 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatecalico-example-0uch4k" } I0803 22:18:18.314244 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:18.314409 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:18.314837 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:18.315039 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:18.315145 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:18.315235 20401 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0803 22:18:18.315286 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0803 22:18:18.315359 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:18.315526 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:18.315650 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:18.318066 20401 executor.go:111] Tasks: 118 done / 124 total; 5 can run W0803 22:18:18.318181 20401 api.go:53] PageSize not implemented W0803 22:18:18.318206 20401 api.go:53] PageSize not implemented I0803 22:18:18.318252 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0803 22:18:18.318265 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:18.318370 20401 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0803 22:18:18.318275 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:18.318449 20401 dnsname.go:121] AliasTarget for "api.privatecalico.example.com" is "api-privatecalico-example-0uch4k.elb.cloudmock.com" I0803 22:18:18.318436 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0803 22:18:18.318596 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0803 22:18:18.318662 20401 executor.go:111] Tasks: 123 done / 124 total; 1 can run I0803 22:18:18.318734 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:18.318882 20401 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0803 22:18:18.322700 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0803 22:18:18.322761 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0803 22:18:18.322799 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0803 22:18:18.322835 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0803 22:18:18.322878 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0803 22:18:18.322920 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0803 22:18:18.322956 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0803 22:18:18.322990 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0803 22:18:18.323027 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0803 22:18:18.323065 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0803 22:18:18.323103 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0803 22:18:18.323143 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0803 22:18:18.323182 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0803 22:18:18.323223 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0803 22:18:18.323261 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0803 22:18:18.323295 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0803 22:18:18.323328 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0803 22:18:18.323365 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0803 22:18:18.323398 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0803 22:18:18.323436 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0803 22:18:18.323473 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0803 22:18:18.323510 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0803 22:18:18.323549 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0803 22:18:18.323584 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0803 22:18:18.323621 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0803 22:18:18.323658 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0803 22:18:18.323694 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0803 22:18:18.323732 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0803 22:18:18.323767 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0803 22:18:18.323800 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0803 22:18:18.323839 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0803 22:18:18.323877 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0803 22:18:18.323916 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0803 22:18:18.325584 20401 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0803 22:18:18.325613 20401 keypairs.go:95] DescribeKeyPairs: { } I0803 22:18:18.325654 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0803 22:18:18.325720 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0803 22:18:18.325799 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0803 22:18:18.325858 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0803 22:18:18.325905 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0803 22:18:18.325963 20401 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0803 22:18:18.326011 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0803 22:18:18.326057 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0803 22:18:18.326118 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0803 22:18:18.326173 20401 routetable.go:53] DescribeRouteTables: { } I0803 22:18:18.326203 20401 natgateway.go:117] DescribeNatGateways: { } I0803 22:18:18.326240 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0803 22:18:18.326280 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } W0803 22:18:18.326336 20401 api.go:53] PageSize not implemented I0803 22:18:18.326354 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0803 22:18:18.326407 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0803 22:18:18.326435 20401 loadbalancers.go:34] PageSize not implemented I0803 22:18:18.326455 20401 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0803 22:18:18.326479 20401 targetgroups.go:35] PageSize not implemented I0803 22:18:18.326498 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0803 22:18:18.326547 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:18.326574 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:18.326616 20401 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0803 22:18:18.326656 20401 iamrole.go:89] ListRoles: { } I0803 22:18:18.326701 20401 internetgateways.go:101] DescribeInternetGateways: { } I0803 22:18:18.326748 20401 routetable.go:53] DescribeRouteTables: { } I0803 22:18:18.326790 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0803 22:18:18.326829 20401 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0803 22:18:18.326866 20401 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } autoscaling-group:nodes.privatecalico.example.com ok I0803 22:18:18.327508 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } autoscaling-group:bastion.privatecalico.example.com ok I0803 22:18:18.327596 20401 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatecalico-example-0uch4k" } load-balancer:api-privatecalico-example-0uch4k ok I0803 22:18:18.327571 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } autoscaling-group:master-us-test-1a.masters.privatecalico.example.com ok W0803 22:18:18.327749 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:18.327684 20401 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } load-balancer:bastion-privatecalico-exa-hocohm ok I0803 22:18:18.327638 20401 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0803 22:18:18.327776 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0803 22:18:18.327734 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.privatecalico.example.com. ok autoscaling-config:lt-3 ok I0803 22:18:18.327864 20401 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } iam-instance-profile:masters.privatecalico.example.com ok I0803 22:18:18.327960 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:18.328013 20401 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0803 22:18:18.328080 20401 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } iam-instance-profile:bastions.privatecalico.example.com ok I0803 22:18:18.328114 20401 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0803 22:18:18.328161 20401 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } iam-instance-profile:nodes.privatecalico.example.com ok I0803 22:18:18.328051 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0803 22:18:18.328205 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:18.328230 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0803 22:18:18.328285 20401 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-1 ok autoscaling-config:lt-2 ok I0803 22:18:18.328394 20401 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0803 22:18:18.328463 20401 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0803 22:18:18.328510 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0803 22:18:18.328566 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W0803 22:18:18.328836 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:18.328863 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0803 22:18:18.328909 20401 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0803 22:18:18.328997 20401 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0803 22:18:18.329040 20401 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0803 22:18:18.329207 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:18.329204 20401 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatecalico.example.com" } I0803 22:18:18.329284 20401 iamrole.go:180] ListAttachedRolePolicies: bastions.privatecalico.example.com I0803 22:18:18.329304 20401 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0803 22:18:18.329340 20401 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatecalico.example.com" } iam-role:bastions.privatecalico.example.com ok I0803 22:18:18.329272 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 } ] } I0803 22:18:18.329371 20401 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatecalico.example.com" } W0803 22:18:18.329423 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:18.329458 20401 iamrole.go:180] ListAttachedRolePolicies: nodes.privatecalico.example.com I0803 22:18:18.329479 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0803 22:18:18.329488 20401 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0803 22:18:18.329522 20401 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatecalico.example.com" } iam-role:nodes.privatecalico.example.com ok I0803 22:18:18.329516 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0803 22:18:18.329551 20401 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatecalico.example.com" } I0803 22:18:18.329610 20401 iamrole.go:180] ListAttachedRolePolicies: masters.privatecalico.example.com I0803 22:18:18.329625 20401 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0803 22:18:18.329568 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } I0803 22:18:18.329655 20401 iamrole.go:137] DeleteRole: { RoleName: "masters.privatecalico.example.com" } iam-role:masters.privatecalico.example.com ok W0803 22:18:18.329675 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:18.329691 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0803 22:18:18.329739 20401 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I0803 22:18:18.329829 20401 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0803 22:18:18.329936 20401 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0803 22:18:18.330038 20401 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0803 22:18:18.330109 20401 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0803 22:18:18.330188 20401 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0803 22:18:18.330274 20401 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0803 22:18:18.330541 20401 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateCalico (1.00s) === RUN TestLifecyclePrivateKopeio I0803 22:18:18.330823 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:18.331068 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:18.331115 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:18.331160 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:18.331208 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:18.331249 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:18.331281 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:18.331321 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:18.331348 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:18.331405 20401 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatekopeio.example.com I0803 22:18:18.364270 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:18.367717 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:18.367874 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:18.367948 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:18.407667 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:18.516510 20401 iamrole.go:89] ListRoles: { } I0803 22:18:18.521539 20401 executor.go:111] Tasks: 0 done / 123 total; 47 can run I0803 22:18:18.522003 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } I0803 22:18:18.522021 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.522199 20401 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "bastions.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0803 22:18:18.522381 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:18.522351 20401 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-main.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0803 22:18:18.522619 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0803 22:18:18.522595 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } I0803 22:18:18.522717 20401 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0803 22:18:18.522706 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.522853 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } I0803 22:18:18.522968 20401 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0803 22:18:18.522962 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.523305 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:18.523327 20401 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0803 22:18:18.523407 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:18.523521 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.523675 20401 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0803 22:18:18.523876 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0803 22:18:18.523979 20401 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0803 22:18:18.524039 20401 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0803 22:18:18.524096 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0803 22:18:18.524167 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:18.524234 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0803 22:18:18.524499 20401 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0803 22:18:18.524717 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0803 22:18:18.525066 20401 executor.go:111] Tasks: 47 done / 123 total; 31 can run I0803 22:18:18.525314 20401 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0803 22:18:18.525215 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.525374 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0803 22:18:18.525553 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.525594 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0803 22:18:18.525536 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0803 22:18:18.525704 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0803 22:18:18.525742 20401 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I0803 22:18:18.525730 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.525770 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.8.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1b.privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0803 22:18:18.525825 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0803 22:18:18.525911 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0803 22:18:18.526000 20401 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0803 22:18:18.525994 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0803 22:18:18.526070 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0803 22:18:18.526087 20401 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0803 22:18:18.526205 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0803 22:18:18.526305 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0803 22:18:18.526199 20401 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0803 22:18:18.526356 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:18.526403 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0803 22:18:18.526449 20401 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0803 22:18:18.526540 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0803 22:18:18.526587 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0803 22:18:18.526635 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0803 22:18:18.526739 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:18:18.526801 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.527046 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.527371 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.527375 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" } ] }], VpcId: "vpc-1" } I0803 22:18:18.527506 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0803 22:18:18.527597 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.527716 20401 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I0803 22:18:18.527764 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0803 22:18:18.527921 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0803 22:18:18.527955 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0803 22:18:18.527710 20401 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0803 22:18:18.528137 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:18.528266 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.528427 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.privatekopeio.example.com" }] }], VpcId: "vpc-1" } I0803 22:18:18.528553 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I0803 22:18:18.528698 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:18.528849 20401 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0803 22:18:18.528977 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0803 22:18:18.529063 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }], VpcId: "vpc-1" } I0803 22:18:18.529192 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0803 22:18:18.529249 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0803 22:18:18.529300 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0803 22:18:18.529349 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0803 22:18:18.529391 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0803 22:18:18.529483 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0803 22:18:18.529537 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0803 22:18:18.529592 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1b"] }] } I0803 22:18:18.529711 20401 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0803 22:18:18.529793 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0803 22:18:18.529864 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0803 22:18:18.529892 20401 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" }, { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0803 22:18:18.530010 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0803 22:18:18.530095 20401 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0803 22:18:18.530149 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0803 22:18:18.530218 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:18.530291 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0803 22:18:18.530384 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0803 22:18:18.530478 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0803 22:18:18.530580 20401 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [ { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" } ] } I0803 22:18:18.530669 20401 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] } I0803 22:18:18.530756 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0803 22:18:18.531625 20401 executor.go:111] Tasks: 78 done / 123 total; 39 can run I0803 22:18:18.531804 20401 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0803 22:18:18.531929 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } W0803 22:18:18.532059 20401 api.go:53] PageSize not implemented W0803 22:18:18.532090 20401 api.go:53] PageSize not implemented I0803 22:18:18.532077 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0803 22:18:18.532228 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0803 22:18:18.532327 20401 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] } I0803 22:18:18.532396 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0803 22:18:18.532434 20401 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0803 22:18:18.532392 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0803 22:18:18.532495 20401 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0803 22:18:18.532580 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0803 22:18:18.532649 20401 tags.go:74] AddTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"], Tags: [{ Key: "Name", Value: "api.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0803 22:18:18.532730 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0803 22:18:18.532627 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0803 22:18:18.532785 20401 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0803 22:18:18.532867 20401 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0803 22:18:18.533015 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:18.533330 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0803 22:18:18.533453 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:18.533583 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:18.533708 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:18.533841 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0803 22:18:18.533905 20401 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0803 22:18:18.534038 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:18.534137 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:18.534269 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0803 22:18:18.534371 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0803 22:18:18.534618 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-3" } I0803 22:18:18.534682 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatekopeio.example.com" }] }] } I0803 22:18:18.534840 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0803 22:18:18.535009 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-udp-1to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0803 22:18:18.535176 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:18.535291 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-2382to4000-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0803 22:18:18.535484 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0803 22:18:18.535657 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0803 22:18:18.535702 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I0803 22:18:18.535861 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0803 22:18:18.535900 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0803 22:18:18.536058 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-nodes.privatekopeio.example.com" }] }] } I0803 22:18:18.536223 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-ingress-tcp-22to22-bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0803 22:18:18.536401 20401 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0803 22:18:18.536448 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } I0803 22:18:18.536549 20401 tags.go:42] CreateTags { Resources: ["nat-b2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0803 22:18:18.536596 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:18.536679 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-4003to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0803 22:18:18.536844 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0803 22:18:18.536970 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-1to2379-masters.privatekopeio.example.com" }] }] } I0803 22:18:18.537100 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0803 22:18:18.537224 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0803 22:18:18.537264 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0803 22:18:18.537373 20401 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0803 22:18:18.537415 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I0803 22:18:18.537522 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-4"] }] } I0803 22:18:18.537562 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0803 22:18:18.537684 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0803 22:18:18.537818 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0803 22:18:18.537942 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0803 22:18:18.538072 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:18.538118 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0803 22:18:18.538157 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0803 22:18:18.538215 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0803 22:18:18.538264 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:18.538312 20401 tags.go:42] CreateTags { Resources: ["nat-a2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0803 22:18:18.538358 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-4" } I0803 22:18:18.538419 20401 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0803 22:18:18.538469 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0803 22:18:18.538514 20401 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0803 22:18:18.538595 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0803 22:18:18.538643 20401 tags.go:42] CreateTags { Resources: ["subnet-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0803 22:18:18.538680 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0803 22:18:18.538722 20401 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0803 22:18:18.539258 20401 executor.go:111] Tasks: 117 done / 123 total; 6 can run I0803 22:18:18.539358 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:18.539447 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:18.539528 20401 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-a2345678", RouteTableId: "rtb-2" } I0803 22:18:18.539592 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0803 22:18:18.539610 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:18.539724 20401 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-b2345678", RouteTableId: "rtb-3" } I0803 22:18:18.539790 20401 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0803 22:18:18.539831 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:18.539870 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:18.539930 20401 dns.go:238] Pre-creating DNS records I0803 22:18:18.539959 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:18.546448 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:18.549537 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:18.549645 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:18.549742 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:18.590720 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:18.700755 20401 iamrole.go:89] ListRoles: { } I0803 22:18:18.703016 20401 executor.go:111] Tasks: 0 done / 123 total; 47 can run I0803 22:18:18.703460 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] }] } I0803 22:18:18.704585 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.704947 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.705140 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0803 22:18:18.705180 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0803 22:18:18.705273 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:18.705418 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:18.705494 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:18.705537 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:18.707052 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.711467 20401 executor.go:111] Tasks: 47 done / 123 total; 31 can run I0803 22:18:18.711640 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0803 22:18:18.711740 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0803 22:18:18.711800 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.711942 20401 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0803 22:18:18.711993 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0803 22:18:18.712028 20401 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I0803 22:18:18.712076 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.712252 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:18:18.712298 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.712429 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0803 22:18:18.712466 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0803 22:18:18.712613 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:18.712665 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.712859 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0803 22:18:18.712909 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:18.712996 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.713143 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:18.713233 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0803 22:18:18.713384 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.713516 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.713638 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I0803 22:18:18.713780 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.713875 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0803 22:18:18.713997 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:18.714053 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0803 22:18:18.714096 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0803 22:18:18.714160 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0803 22:18:18.716849 20401 executor.go:111] Tasks: 78 done / 123 total; 39 can run W0803 22:18:18.717010 20401 api.go:53] PageSize not implemented I0803 22:18:18.717046 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0803 22:18:18.717149 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0803 22:18:18.717189 20401 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0803 22:18:18.717187 20401 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0803 22:18:18.717295 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0803 22:18:18.717361 20401 api.go:53] PageSize not implemented I0803 22:18:18.717426 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:18.717434 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0803 22:18:18.717554 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0803 22:18:18.717582 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:18.717602 20401 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0803 22:18:18.717828 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:18.717943 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:18.718123 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:18.718342 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:18.718454 20401 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0803 22:18:18.718555 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0803 22:18:18.718644 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:18.718730 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:18.718881 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:18.719097 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:18.721283 20401 executor.go:111] Tasks: 117 done / 123 total; 6 can run I0803 22:18:18.721384 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:18.721469 20401 dnsname.go:121] AliasTarget for "api.privatekopeio.example.com" is "api-privatekopeio-example-tl2bv8.elb.cloudmock.com" I0803 22:18:18.721496 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0803 22:18:18.721459 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } W0803 22:18:18.721545 20401 api.go:53] PageSize not implemented I0803 22:18:18.721584 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0803 22:18:18.721620 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0803 22:18:18.721779 20401 api.go:53] PageSize not implemented I0803 22:18:18.721812 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8","bastion-privatekopeio-exa-d8ef8e"] } I0803 22:18:18.722035 20401 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0803 22:18:18.725892 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0803 22:18:18.725959 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0803 22:18:18.725999 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0803 22:18:18.726038 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0803 22:18:18.726078 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0803 22:18:18.726115 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0803 22:18:18.726149 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0803 22:18:18.726184 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0803 22:18:18.726221 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0803 22:18:18.726259 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0803 22:18:18.726306 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0803 22:18:18.726347 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0803 22:18:18.726385 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0803 22:18:18.726422 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0803 22:18:18.726457 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0803 22:18:18.726496 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0803 22:18:18.726532 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0803 22:18:18.726572 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0803 22:18:18.726610 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0803 22:18:18.726648 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0803 22:18:18.726685 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0803 22:18:18.726721 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0803 22:18:18.726758 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0803 22:18:18.726797 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0803 22:18:18.726836 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0803 22:18:18.726871 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0803 22:18:18.726912 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0803 22:18:18.726967 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0803 22:18:18.727006 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0803 22:18:18.727040 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0803 22:18:18.727074 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0803 22:18:18.727113 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0803 22:18:18.727152 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0803 22:18:18.727191 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0803 22:18:18.728949 20401 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0803 22:18:18.728977 20401 keypairs.go:95] DescribeKeyPairs: { } I0803 22:18:18.729028 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.729100 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0803 22:18:18.729183 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.729241 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.729290 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.729347 20401 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.729390 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.729442 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0803 22:18:18.729514 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.729586 20401 routetable.go:53] DescribeRouteTables: { } I0803 22:18:18.729618 20401 natgateway.go:117] DescribeNatGateways: { } I0803 22:18:18.729656 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0803 22:18:18.729699 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } W0803 22:18:18.729753 20401 api.go:53] PageSize not implemented I0803 22:18:18.729771 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8","bastion-privatekopeio-exa-d8ef8e"] } I0803 22:18:18.729824 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0803 22:18:18.729852 20401 loadbalancers.go:34] PageSize not implemented I0803 22:18:18.729872 20401 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0803 22:18:18.729896 20401 targetgroups.go:35] PageSize not implemented I0803 22:18:18.729915 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0803 22:18:18.729970 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:18.730000 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:18.730055 20401 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0803 22:18:18.730092 20401 iamrole.go:89] ListRoles: { } I0803 22:18:18.730137 20401 internetgateways.go:101] DescribeInternetGateways: { } I0803 22:18:18.730185 20401 routetable.go:53] DescribeRouteTables: { } I0803 22:18:18.730235 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2","rtb-3"] } I0803 22:18:18.730278 20401 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0803 22:18:18.730310 20401 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } autoscaling-config:lt-1 ok autoscaling-group:master-us-test-1a.masters.privatekopeio.example.com ok autoscaling-group:bastion.privatekopeio.example.com ok I0803 22:18:18.731067 20401 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } autoscaling-config:lt-3 ok I0803 22:18:18.731071 20401 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } load-balancer:bastion-privatekopeio-exa-d8ef8e ok I0803 22:18:18.731125 20401 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } iam-instance-profile:nodes.privatekopeio.example.com ok autoscaling-group:nodes.privatekopeio.example.com ok I0803 22:18:18.731146 20401 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0803 22:18:18.731172 20401 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } autoscaling-config:lt-2 ok I0803 22:18:18.731226 20401 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } iam-instance-profile:masters.privatekopeio.example.com ok I0803 22:18:18.731265 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:18.731293 20401 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0803 22:18:18.731340 20401 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } iam-instance-profile:bastions.privatekopeio.example.com ok I0803 22:18:18.731149 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.privatekopeio.example.com ok I0803 22:18:18.731373 20401 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } load-balancer:api-privatekopeio-example-tl2bv8 ok I0803 22:18:18.731330 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0803 22:18:18.731490 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:18.731517 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0803 22:18:18.731560 20401 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0803 22:18:18.731602 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0803 22:18:18.731660 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] } ] } W0803 22:18:18.731871 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:18.731898 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0803 22:18:18.731944 20401 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0803 22:18:18.732001 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:18.732060 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W0803 22:18:18.732181 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:18.732195 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } I0803 22:18:18.732226 20401 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } security-group:sg-1 ok keypair:key-1 ok I0803 22:18:18.732282 20401 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0803 22:18:18.732420 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0803 22:18:18.732475 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } I0803 22:18:18.732496 20401 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatekopeio.example.com" } W0803 22:18:18.732563 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:18.732605 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0803 22:18:18.732582 20401 iamrole.go:180] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0803 22:18:18.732668 20401 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-4" } subnet:subnet-4 ok I0803 22:18:18.732678 20401 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0803 22:18:18.732729 20401 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatekopeio.example.com" } iam-role:nodes.privatekopeio.example.com ok I0803 22:18:18.732732 20401 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0803 22:18:18.732776 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0803 22:18:18.732776 20401 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatekopeio.example.com" } I0803 22:18:18.732834 20401 iamrole.go:180] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0803 22:18:18.732851 20401 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0803 22:18:18.732882 20401 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatekopeio.example.com" } iam-role:bastions.privatekopeio.example.com ok I0803 22:18:18.732826 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W0803 22:18:18.732926 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:18.732915 20401 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatekopeio.example.com" } I0803 22:18:18.732940 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0803 22:18:18.732958 20401 iamrole.go:180] ListAttachedRolePolicies: masters.privatekopeio.example.com I0803 22:18:18.732972 20401 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0803 22:18:18.732985 20401 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I0803 22:18:18.733002 20401 iamrole.go:137] DeleteRole: { RoleName: "masters.privatekopeio.example.com" } iam-role:masters.privatekopeio.example.com ok I0803 22:18:18.733040 20401 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0803 22:18:18.733134 20401 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-3" } route-table:rtb-3 ok I0803 22:18:18.733195 20401 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0803 22:18:18.733251 20401 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0803 22:18:18.733330 20401 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0803 22:18:18.733412 20401 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0803 22:18:18.733721 20401 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateKopeio (0.40s) === RUN TestLifecycleIPv6 I0803 22:18:18.734086 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:18.734371 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:18.734419 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:18.734462 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:18.734509 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:18.734549 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:18.734583 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:18.734622 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:18.734651 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:18.734704 20401 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal-ipv6.example.com I0803 22:18:18.760567 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:18.763874 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:18.763970 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:18.803713 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:18.985408 20401 iamrole.go:89] ListRoles: { } I0803 22:18:18.987504 20401 executor.go:111] Tasks: 0 done / 104 total; 45 can run I0803 22:18:18.988005 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0803 22:18:18.988134 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0803 22:18:18.988358 20401 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0803 22:18:18.988508 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0803 22:18:18.988570 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:18.988510 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } I0803 22:18:18.988537 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:18.988627 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:18.988658 20401 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal-ipv6.example.com" }] } I0803 22:18:18.988805 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0803 22:18:18.988882 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } I0803 22:18:18.988991 20401 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0803 22:18:18.989076 20401 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0803 22:18:18.989254 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0803 22:18:18.989318 20401 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0803 22:18:18.989443 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0803 22:18:18.989494 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0803 22:18:18.989678 20401 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0803 22:18:18.989740 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "minimal-ipv6.example.com" }] }] } I0803 22:18:18.989959 20401 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0803 22:18:18.990000 20401 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0803 22:18:18.990027 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0803 22:18:18.990072 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:18.991399 20401 executor.go:111] Tasks: 45 done / 104 total; 21 can run I0803 22:18:18.991722 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0803 22:18:18.991626 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0803 22:18:18.991803 20401 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0803 22:18:18.991778 20401 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0803 22:18:18.991906 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:18.992007 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0803 22:18:18.992061 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0803 22:18:18.992221 20401 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-minimal-ipv6-example--bne5ih", Port: 443, Protocol: "TCP", Tags: [{ Key: "Name", Value: "tcp-minimal-ipv6-example--bne5ih" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0803 22:18:18.992327 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0803 22:18:18.992454 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0803 22:18:18.992773 20401 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0803 22:18:18.992931 20401 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], VpcId: "vpc-1" } I0803 22:18:18.992952 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0803 22:18:18.993382 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0803 22:18:18.993924 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0803 22:18:18.994264 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0803 22:18:18.994525 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public-us-test-1a"] }] } I0803 22:18:18.994849 20401 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0803 22:18:18.994969 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:18.995057 20401 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0803 22:18:18.995111 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }], VpcId: "vpc-1" } I0803 22:18:18.995209 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0803 22:18:18.995295 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0803 22:18:18.995381 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0803 22:18:18.995323 20401 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" }, { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0803 22:18:18.995466 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0803 22:18:18.996233 20401 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0803 22:18:18.996291 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0803 22:18:18.996342 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0803 22:18:18.996413 20401 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal-ipv6.example.com" } ] } I0803 22:18:18.996497 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0803 22:18:18.996546 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0803 22:18:18.996591 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0803 22:18:18.996643 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:18.996729 20401 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" }, { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" } ] } I0803 22:18:18.996829 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:18.996926 20401 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0803 22:18:18.999945 20401 executor.go:111] Tasks: 66 done / 104 total; 30 can run I0803 22:18:19.000082 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:19.000248 20401 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0803 22:18:19.000311 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-1to2379-masters.minimal-ipv6.example.com" }] }] } I0803 22:18:19.000574 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0803 22:18:19.000816 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", Ipv6CidrBlock: "2001:db8:0:111::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0803 22:18:19.000878 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-4003to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0803 22:18:19.001154 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0803 22:18:19.001334 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-2382to4000-masters.minimal-ipv6.example.com" }] }] } I0803 22:18:19.001485 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0803 22:18:19.001658 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:19.001737 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:19.001813 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:19.001898 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:19.001968 20401 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0803 22:18:19.001968 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0803 22:18:19.002170 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.002297 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.002363 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0803 22:18:19.002522 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0803 22:18:19.002663 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0803 22:18:19.002811 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-udp-1to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0803 22:18:19.002963 20401 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0803 22:18:19.003420 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0803 22:18:19.003601 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.003716 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.003781 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0803 22:18:19.003928 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0803 22:18:19.004167 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0803 22:18:19.004414 20401 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0803 22:18:19.004509 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0803 22:18:19.004768 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" }] }] } I0803 22:18:19.004982 20401 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0803 22:18:19.005030 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0803 22:18:19.005174 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0803 22:18:19.005317 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-::/0" }] }] } I0803 22:18:19.005489 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0803 22:18:19.005628 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0803 22:18:19.005688 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:18:19.006734 20401 executor.go:111] Tasks: 96 done / 104 total; 5 can run I0803 22:18:19.006824 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:19.006910 20401 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0803 22:18:19.006948 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0803 22:18:19.007043 20401 loadbalancers.go:34] PageSize not implemented I0803 22:18:19.007001 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:19.007040 20401 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" }] }] } I0803 22:18:19.007176 20401 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "dualstack", Name: "api-minimal-ipv6-example--jhj9te", SubnetMappings: [{ SubnetId: "subnet-1" }], Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "api.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }], Type: "network" } I0803 22:18:19.007265 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0803 22:18:19.007373 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0803 22:18:19.007352 20401 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1", Port: 443, Protocol: "TCP" } I0803 22:18:19.007458 20401 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0803 22:18:19.007539 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0803 22:18:19.007587 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0803 22:18:19.007511 20401 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "false" },{ Key: "access_logs.s3.enabled", Value: "false" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0803 22:18:19.007664 20401 executor.go:111] Tasks: 101 done / 104 total; 3 can run I0803 22:18:19.007754 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:19.007869 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:19.007911 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:19.008014 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:19.008065 20401 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "64:ff9b::/96", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0803 22:18:19.008142 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:19.008237 20401 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0803 22:18:19.008278 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:19.008311 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:19.008366 20401 dns.go:238] Pre-creating DNS records I0803 22:18:19.008392 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:19.014127 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:19.017411 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.017511 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.056585 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:19.229334 20401 iamrole.go:89] ListRoles: { } I0803 22:18:19.231128 20401 executor.go:111] Tasks: 0 done / 104 total; 45 can run I0803 22:18:19.231351 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0803 22:18:19.231552 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0803 22:18:19.231713 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0803 22:18:19.231825 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0803 22:18:19.231883 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0803 22:18:19.231971 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:19.232035 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:19.232042 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0803 22:18:19.232308 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:19.232417 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:19.233790 20401 executor.go:111] Tasks: 45 done / 104 total; 21 can run I0803 22:18:19.234034 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0803 22:18:19.234108 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0803 22:18:19.234220 20401 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0803 22:18:19.234316 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0803 22:18:19.234373 20401 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0803 22:18:19.234297 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0803 22:18:19.234471 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:19.234527 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0803 22:18:19.234667 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0803 22:18:19.234714 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0803 22:18:19.234834 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:19.234926 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0803 22:18:19.235047 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0803 22:18:19.235203 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:19.235298 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:19.243636 20401 executor.go:111] Tasks: 66 done / 104 total; 30 can run I0803 22:18:19.243826 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:19.243987 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:19.244119 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.244333 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.244640 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.244833 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.245104 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0803 22:18:19.245298 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:18:19.245375 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:19.245489 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:19.245565 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:19.245680 20401 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0803 22:18:19.245723 20401 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0803 22:18:19.245759 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0803 22:18:19.246851 20401 executor.go:111] Tasks: 96 done / 104 total; 5 can run I0803 22:18:19.246986 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0803 22:18:19.247111 20401 loadbalancers.go:34] PageSize not implemented I0803 22:18:19.247055 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:19.247136 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0803 22:18:19.247173 20401 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0803 22:18:19.247186 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0803 22:18:19.247222 20401 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0803 22:18:19.247269 20401 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0803 22:18:19.247257 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:19.247304 20401 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0803 22:18:19.247427 20401 executor.go:111] Tasks: 101 done / 104 total; 3 can run I0803 22:18:19.247511 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:19.247607 20401 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0803 22:18:19.247634 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0803 22:18:19.247659 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0803 22:18:19.247631 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:19.247552 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:19.247718 20401 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0803 22:18:19.247742 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0803 22:18:19.247771 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0803 22:18:19.247858 20401 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0803 22:18:19.251187 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0803 22:18:19.251250 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0803 22:18:19.251290 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0803 22:18:19.251329 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0803 22:18:19.251367 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0803 22:18:19.251407 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0803 22:18:19.251442 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0803 22:18:19.251477 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0803 22:18:19.251513 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0803 22:18:19.251553 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0803 22:18:19.251591 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0803 22:18:19.251630 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0803 22:18:19.251677 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0803 22:18:19.251715 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0803 22:18:19.251749 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0803 22:18:19.251784 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0803 22:18:19.251822 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0803 22:18:19.251855 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0803 22:18:19.251891 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0803 22:18:19.251928 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0803 22:18:19.251963 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0803 22:18:19.251999 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0803 22:18:19.252036 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0803 22:18:19.252070 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0803 22:18:19.252121 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0803 22:18:19.252158 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0803 22:18:19.252197 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0803 22:18:19.252234 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0803 22:18:19.252271 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0803 22:18:19.254517 20401 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0803 22:18:19.254546 20401 keypairs.go:95] DescribeKeyPairs: { } I0803 22:18:19.254584 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0803 22:18:19.254640 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0803 22:18:19.254704 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0803 22:18:19.254766 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0803 22:18:19.254818 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0803 22:18:19.254870 20401 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0803 22:18:19.254912 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0803 22:18:19.254956 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0803 22:18:19.255011 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0803 22:18:19.255058 20401 routetable.go:53] DescribeRouteTables: { } I0803 22:18:19.255085 20401 natgateway.go:117] DescribeNatGateways: { } I0803 22:18:19.255127 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0803 22:18:19.255169 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } W0803 22:18:19.255226 20401 api.go:53] PageSize not implemented I0803 22:18:19.255245 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0803 22:18:19.255274 20401 loadbalancers.go:34] PageSize not implemented I0803 22:18:19.255288 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0803 22:18:19.255329 20401 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0803 22:18:19.255353 20401 targetgroups.go:35] PageSize not implemented I0803 22:18:19.255371 20401 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" ] } I0803 22:18:19.255449 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:19.255481 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:19.255526 20401 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0803 22:18:19.255563 20401 iamrole.go:89] ListRoles: { } I0803 22:18:19.255604 20401 internetgateways.go:101] DescribeInternetGateways: { } I0803 22:18:19.255654 20401 routetable.go:53] DescribeRouteTables: { } I0803 22:18:19.255696 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I0803 22:18:19.255736 20401 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0803 22:18:19.255770 20401 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0803 22:18:19.256351 20401 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0803 22:18:19.256421 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:19.256422 20401 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1 ok autoscaling-group:master-us-test-1a.masters.minimal-ipv6.example.com ok I0803 22:18:19.256528 20401 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0803 22:18:19.256561 20401 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } autoscaling-group:nodes.minimal-ipv6.example.com ok I0803 22:18:19.256610 20401 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } iam-instance-profile:nodes.minimal-ipv6.example.com ok target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4 ok I0803 22:18:19.256464 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [ { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }, { Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } } ] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/AAAA/api.minimal-ipv6.example.com ok I0803 22:18:19.256659 20401 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0803 22:18:19.256477 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 } ] } W0803 22:18:19.256776 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:19.256772 20401 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } iam-instance-profile:masters.minimal-ipv6.example.com ok I0803 22:18:19.256795 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0803 22:18:19.256832 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:19.256890 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0803 22:18:19.256950 20401 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0803 22:18:19.257065 20401 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok autoscaling-config:lt-1 ok I0803 22:18:19.257172 20401 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0803 22:18:19.257274 20401 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0803 22:18:19.257327 20401 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-2 ok I0803 22:18:19.257409 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0803 22:18:19.257468 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 } ] } W0803 22:18:19.257616 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:19.257631 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0803 22:18:19.257674 20401 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0803 22:18:19.257777 20401 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I0803 22:18:19.257842 20401 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0803 22:18:19.257835 20401 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal-ipv6.example.com" } I0803 22:18:19.257919 20401 iamrole.go:180] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0803 22:18:19.257940 20401 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0803 22:18:19.257974 20401 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal-ipv6.example.com" } iam-role:masters.minimal-ipv6.example.com ok I0803 22:18:19.258007 20401 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal-ipv6.example.com" } I0803 22:18:19.258061 20401 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0803 22:18:19.258075 20401 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0803 22:18:19.258106 20401 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal-ipv6.example.com" } iam-role:nodes.minimal-ipv6.example.com ok I0803 22:18:19.258172 20401 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0803 22:18:19.258258 20401 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0803 22:18:19.258348 20401 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0803 22:18:19.258619 20401 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleIPv6 (0.52s) === RUN TestLifecycleSharedVPC I0803 22:18:19.258920 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:19.259204 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:19.259250 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:19.259293 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:19.259338 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:19.259376 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:19.259408 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:19.259443 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:19.259471 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:19.259527 20401 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedvpc.example.com I0803 22:18:19.283198 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:19.286048 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.286138 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.324710 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:19.422207 20401 iamrole.go:89] ListRoles: { } I0803 22:18:19.423726 20401 executor.go:111] Tasks: 0 done / 82 total; 42 can run I0803 22:18:19.423985 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:19.424047 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0803 22:18:19.424176 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0803 22:18:19.424217 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0803 22:18:19.424272 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:19.424407 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0803 22:18:19.424596 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } I0803 22:18:19.424685 20401 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0803 22:18:19.424720 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:19.424733 20401 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0803 22:18:19.425003 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0803 22:18:19.424945 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } I0803 22:18:19.425104 20401 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0803 22:18:19.425074 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0803 22:18:19.425370 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0803 22:18:19.425695 20401 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0803 22:18:19.425957 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0803 22:18:19.427184 20401 executor.go:111] Tasks: 42 done / 82 total; 17 can run I0803 22:18:19.427448 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0803 22:18:19.427580 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0803 22:18:19.427680 20401 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I0803 22:18:19.427756 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0803 22:18:19.427743 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0803 22:18:19.427827 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0803 22:18:19.427960 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0803 22:18:19.427973 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0803 22:18:19.428092 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0803 22:18:19.428148 20401 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0803 22:18:19.428173 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0803 22:18:19.428213 20401 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-12345678" } I0803 22:18:19.428409 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0803 22:18:19.428459 20401 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0803 22:18:19.428456 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-12345678" } I0803 22:18:19.428547 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0803 22:18:19.428642 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0803 22:18:19.428694 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:19.428747 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0803 22:18:19.428872 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0803 22:18:19.428937 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:19.428993 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0803 22:18:19.429040 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:18:19.429740 20401 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0803 22:18:19.432611 20401 executor.go:111] Tasks: 59 done / 82 total; 21 can run I0803 22:18:19.432825 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0803 22:18:19.433014 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.433137 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.433198 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0803 22:18:19.433493 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedvpc.example.com" }] }] } I0803 22:18:19.433886 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0803 22:18:19.434114 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:19.434231 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:19.434346 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:19.434430 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0803 22:18:19.434635 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-1to2379-masters.sharedvpc.example.com" }] }] } I0803 22:18:19.434815 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.434925 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.435102 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0803 22:18:19.435280 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-udp-1to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0803 22:18:19.435446 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0803 22:18:19.435596 20401 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0803 22:18:19.435655 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" }] }] } I0803 22:18:19.435798 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-2382to4000-masters.sharedvpc.example.com" }] }] } I0803 22:18:19.435942 20401 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0803 22:18:19.435992 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-4003to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0803 22:18:19.436130 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0803 22:18:19.436179 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0803 22:18:19.436220 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0803 22:18:19.436353 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0803 22:18:19.437405 20401 executor.go:111] Tasks: 80 done / 82 total; 2 can run I0803 22:18:19.438581 20401 executor.go:111] Tasks: 82 done / 82 total; 0 can run I0803 22:18:19.438697 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:19.438781 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:19.438858 20401 dns.go:238] Pre-creating DNS records I0803 22:18:19.438908 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:19.446807 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:19.449759 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.449901 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.489367 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:19.585305 20401 iamrole.go:89] ListRoles: { } I0803 22:18:19.586858 20401 executor.go:111] Tasks: 0 done / 82 total; 42 can run I0803 22:18:19.587437 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:19.587500 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0803 22:18:19.587693 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] }] } I0803 22:18:19.587882 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0803 22:18:19.587984 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0803 22:18:19.588030 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0803 22:18:19.588094 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:19.588170 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:19.591272 20401 executor.go:111] Tasks: 42 done / 82 total; 17 can run I0803 22:18:19.591563 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0803 22:18:19.591968 20401 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I0803 22:18:19.592056 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:19.592397 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0803 22:18:19.592233 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0803 22:18:19.592784 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0803 22:18:19.593345 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0803 22:18:19.595375 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0803 22:18:19.595573 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:18:19.595620 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:19.599877 20401 executor.go:111] Tasks: 59 done / 82 total; 21 can run I0803 22:18:19.600204 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.600392 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.600539 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:19.600733 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.600907 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.601065 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:19.601327 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:19.603487 20401 executor.go:111] Tasks: 80 done / 82 total; 2 can run I0803 22:18:19.603701 20401 executor.go:111] Tasks: 82 done / 82 total; 0 can run I0803 22:18:19.606550 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0803 22:18:19.606615 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0803 22:18:19.606653 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0803 22:18:19.606686 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0803 22:18:19.606723 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0803 22:18:19.606757 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0803 22:18:19.606793 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0803 22:18:19.606833 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0803 22:18:19.606872 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0803 22:18:19.606915 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0803 22:18:19.606947 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0803 22:18:19.606979 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0803 22:18:19.607013 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0803 22:18:19.607045 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0803 22:18:19.607082 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0803 22:18:19.607117 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0803 22:18:19.607154 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0803 22:18:19.607187 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0803 22:18:19.607220 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0803 22:18:19.607253 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0803 22:18:19.607289 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0803 22:18:19.607324 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0803 22:18:19.608699 20401 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0803 22:18:19.608727 20401 keypairs.go:95] DescribeKeyPairs: { } I0803 22:18:19.608786 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0803 22:18:19.608841 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0803 22:18:19.608900 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0803 22:18:19.608960 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0803 22:18:19.609005 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0803 22:18:19.609049 20401 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0803 22:18:19.609089 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0803 22:18:19.609129 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0803 22:18:19.609176 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0803 22:18:19.609235 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0803 22:18:19.609270 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } W0803 22:18:19.609319 20401 api.go:53] PageSize not implemented I0803 22:18:19.609342 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0803 22:18:19.609370 20401 loadbalancers.go:34] PageSize not implemented I0803 22:18:19.609390 20401 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0803 22:18:19.609414 20401 targetgroups.go:35] PageSize not implemented I0803 22:18:19.609432 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0803 22:18:19.609499 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:19.609529 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:19.609574 20401 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0803 22:18:19.609616 20401 iamrole.go:89] ListRoles: { } I0803 22:18:19.609657 20401 internetgateways.go:101] DescribeInternetGateways: { } I0803 22:18:19.609697 20401 routetable.go:53] DescribeRouteTables: { } I0803 22:18:19.609728 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-config:lt-2 ok autoscaling-config:lt-1 ok I0803 22:18:19.610186 20401 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } autoscaling-group:master-us-test-1a.masters.sharedvpc.example.com ok I0803 22:18:19.610192 20401 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } I0803 22:18:19.610247 20401 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } volume:vol-2 ok iam-instance-profile:masters.sharedvpc.example.com ok autoscaling-group:nodes.sharedvpc.example.com ok I0803 22:18:19.610291 20401 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0803 22:18:19.610336 20401 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } iam-instance-profile:nodes.sharedvpc.example.com ok I0803 22:18:19.610314 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:19.610294 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.sharedvpc.example.com. ok I0803 22:18:19.610396 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0803 22:18:19.610638 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:19.610664 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0803 22:18:19.610715 20401 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0803 22:18:19.610775 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:19.610829 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0803 22:18:19.610949 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:19.610964 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0803 22:18:19.610999 20401 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0803 22:18:19.611111 20401 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedvpc.example.com" } I0803 22:18:19.611167 20401 iamrole.go:180] ListAttachedRolePolicies: masters.sharedvpc.example.com I0803 22:18:19.611147 20401 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0803 22:18:19.611183 20401 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0803 22:18:19.611254 20401 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedvpc.example.com" } iam-role:masters.sharedvpc.example.com ok I0803 22:18:19.611299 20401 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedvpc.example.com" } I0803 22:18:19.611352 20401 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0803 22:18:19.611367 20401 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0803 22:18:19.611397 20401 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedvpc.example.com" } iam-role:nodes.sharedvpc.example.com ok I0803 22:18:19.611467 20401 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0803 22:18:19.611754 20401 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedVPC (0.35s) === RUN TestLifecycleComplex I0803 22:18:19.612021 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:19.612292 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:19.612339 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:19.612379 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:19.612426 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:19.612467 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:19.612497 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:19.612534 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:19.612562 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:19.612616 20401 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster complex.example.com I0803 22:18:19.647600 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:19.651465 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.651566 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.690612 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:19.793281 20401 iamrole.go:89] ListRoles: { } I0803 22:18:19.795527 20401 executor.go:111] Tasks: 0 done / 116 total; 47 can run I0803 22:18:19.795727 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0803 22:18:19.796058 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0803 22:18:19.796346 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:19.796472 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0803 22:18:19.796574 20401 api.go:53] PageSize not implemented I0803 22:18:19.796649 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } I0803 22:18:19.796638 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:19.796721 20401 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "complex.example.com" } ] }] } I0803 22:18:19.796773 20401 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com", Tags: [ { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] } I0803 22:18:19.796882 20401 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0803 22:18:19.796925 20401 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0803 22:18:19.796907 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } I0803 22:18:19.796965 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0803 22:18:19.796975 20401 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] } I0803 22:18:19.797024 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0803 22:18:19.797263 20401 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "k8s.io/etcd/events", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-events.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "foo/bar", Value: "fib+baz" } ] }], Throughput: 125, VolumeType: "gp3" } I0803 22:18:19.797566 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0803 22:18:19.797651 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0803 22:18:19.797769 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:19.797972 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:19.798142 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:19.798208 20401 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0803 22:18:19.798452 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0803 22:18:19.798547 20401 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "a.etcd-main.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/main", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0803 22:18:19.798725 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0803 22:18:19.798916 20401 executor.go:111] Tasks: 47 done / 116 total; 27 can run I0803 22:18:19.799183 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:19.799225 20401 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0803 22:18:19.799291 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0803 22:18:19.799398 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0803 22:18:19.799322 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:19.799405 20401 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0803 22:18:19.799503 20401 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0803 22:18:19.799478 20401 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0803 22:18:19.799563 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0803 22:18:19.799603 20401 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-complex-example-com-vpjolq", Port: 443, Protocol: "TCP", Tags: [ { Key: "Name", Value: "tcp-complex-example-com-vpjolq" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0803 22:18:19.799614 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-private.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0803 22:18:19.799599 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:19.799754 20401 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tls-complex-example-com-5nursn", Port: 443, Protocol: "TLS", Tags: [ { Key: "Name", Value: "tls-complex-example-com-5nursn" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0803 22:18:19.799946 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:19.799957 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0803 22:18:19.800173 20401 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }] } I0803 22:18:19.800225 20401 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0803 22:18:19.800298 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0803 22:18:19.800374 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0803 22:18:19.800424 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0803 22:18:19.800465 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:19.800611 20401 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0803 22:18:19.800668 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:19.801804 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:19.801951 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:19.802239 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0803 22:18:19.802429 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:19.802520 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:19.802658 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0803 22:18:19.802856 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0803 22:18:19.803045 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0803 22:18:19.802923 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.96.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-utility.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0803 22:18:19.803168 20401 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.1.0.0/16", VpcId: "vpc-1" } I0803 22:18:19.803253 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0803 22:18:19.803344 20401 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I0803 22:18:19.803554 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0803 22:18:19.803514 20401 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I0803 22:18:19.803661 20401 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] } I0803 22:18:19.803297 20401 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0803 22:18:19.803840 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Name", Value: "api-elb.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0803 22:18:19.804086 20401 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.2.0.0/16", VpcId: "vpc-1" } I0803 22:18:19.804173 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I0803 22:18:19.804849 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0803 22:18:19.804967 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I0803 22:18:19.805086 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0803 22:18:19.805148 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:19.805199 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0803 22:18:19.805255 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0803 22:18:19.805297 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0803 22:18:19.805346 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:19.805387 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0803 22:18:19.805469 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0803 22:18:19.805522 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0803 22:18:19.805576 20401 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] } I0803 22:18:19.806439 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:18:19.808090 20401 executor.go:111] Tasks: 74 done / 116 total; 39 can run I0803 22:18:19.808224 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:19.808380 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } I0803 22:18:19.808374 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0803 22:18:19.808467 20401 loadbalancers.go:34] PageSize not implemented I0803 22:18:19.808550 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:19.808621 20401 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "api-complex-example-com-vd3t5n", SubnetMappings: [{ AllocationId: "eipalloc-012345a678b9cdefa", SubnetId: "subnet-2" }], Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "api.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ], Type: "network" } I0803 22:18:19.808869 20401 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-test-1:000000000000:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0803 22:18:19.808981 20401 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0803 22:18:19.808792 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0803 22:18:19.809051 20401 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [ { Key: "load_balancing.cross_zone.enabled", Value: "true" }, { Key: "access_logs.s3.enabled", Value: "true" }, { Key: "access_logs.s3.bucket", Value: "access-log-example" }, { Key: "access_logs.s3.prefix", Value: "" } ], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0803 22:18:19.809080 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-1to2379-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0803 22:18:19.809251 20401 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0803 22:18:19.809389 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }] } I0803 22:18:19.809499 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:19.809574 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0803 22:18:19.809734 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0803 22:18:19.809775 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0803 22:18:19.809877 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0803 22:18:19.810042 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0803 22:18:19.810196 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0803 22:18:19.810237 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:19.810312 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.complex.example.com" } ] }] } I0803 22:18:19.810466 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:19.810538 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0803 22:18:19.810700 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:19.810771 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }] } I0803 22:18:19.810874 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0803 22:18:19.811023 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0803 22:18:19.811114 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }] } I0803 22:18:19.811214 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0803 22:18:19.811359 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0803 22:18:19.811449 20401 routetable.go:149] CreateRoute: { DestinationCidrBlock: "192.168.1.10/32", RouteTableId: "rtb-2", TransitGatewayId: "tgw-0123456" } I0803 22:18:19.811494 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0803 22:18:19.811531 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.811621 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.0/24-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0803 22:18:19.811771 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }] } I0803 22:18:19.811862 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0803 22:18:19.812003 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }] } I0803 22:18:19.812117 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0803 22:18:19.812163 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-4003to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0803 22:18:19.812320 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.812401 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0803 22:18:19.812491 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0803 22:18:19.812632 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-2382to4000-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0803 22:18:19.812781 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0803 22:18:19.812822 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0803 22:18:19.812967 20401 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0803 22:18:19.813008 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0803 22:18:19.813100 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-udp-1to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0803 22:18:19.813253 20401 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", RouteTableId: "rtb-2", TransitGatewayId: "tgw-123456" } I0803 22:18:19.813295 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0803 22:18:19.813455 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.813507 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.813557 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-3" } I0803 22:18:19.816389 20401 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0803 22:18:19.816515 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:19.816690 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:19.816824 20401 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0803 22:18:19.816868 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:19.816900 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:19.816950 20401 dns.go:238] Pre-creating DNS records I0803 22:18:19.816974 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:19.824709 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:19.828585 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.828693 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.867149 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:19.962591 20401 iamrole.go:89] ListRoles: { } I0803 22:18:19.964988 20401 executor.go:111] Tasks: 0 done / 116 total; 47 can run I0803 22:18:19.965361 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:19.965483 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0803 22:18:19.965658 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } W0803 22:18:19.965889 20401 api.go:53] PageSize not implemented I0803 22:18:19.965870 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0803 22:18:19.966054 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0803 22:18:19.966203 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0803 22:18:19.966337 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0803 22:18:19.966471 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:19.966607 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0803 22:18:19.966654 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0803 22:18:19.966757 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:19.966829 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] }] } I0803 22:18:19.968404 20401 executor.go:111] Tasks: 47 done / 116 total; 27 can run I0803 22:18:19.968549 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:19.968684 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0803 22:18:19.968824 20401 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0803 22:18:19.968872 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0803 22:18:19.968887 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0803 22:18:19.968979 20401 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0803 22:18:19.968884 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:19.968968 20401 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0803 22:18:19.969159 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0803 22:18:19.969135 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:19.969301 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I0803 22:18:19.969521 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:19.969770 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0803 22:18:19.969837 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0803 22:18:19.970044 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] }] } I0803 22:18:19.970227 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:19.970359 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:19.970487 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:19.970660 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0803 22:18:19.970717 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0803 22:18:19.970899 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I0803 22:18:19.971070 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:19.971189 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:18:19.971272 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0803 22:18:19.971426 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:19.974262 20401 executor.go:111] Tasks: 74 done / 116 total; 39 can run I0803 22:18:19.974487 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:19.974605 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:19.974751 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:19.974763 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0803 22:18:19.974853 20401 loadbalancers.go:34] PageSize not implemented I0803 22:18:19.974873 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:19.974921 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:19.974960 20401 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0803 22:18:19.975001 20401 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0803 22:18:19.974976 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:19.975038 20401 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0803 22:18:19.975070 20401 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0803 22:18:19.975286 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.975471 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.975998 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:19.976126 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:19.976240 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.976404 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:19.976457 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:19.978842 20401 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0803 22:18:19.978967 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:19.979068 20401 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0803 22:18:19.979093 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0803 22:18:19.979116 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:19.979214 20401 executor.go:111] Tasks: 116 done / 116 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.sslCertificate=arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678] I0803 22:18:19.985402 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:19.988910 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.000659 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:20.004861 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.004962 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.042935 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:20.146637 20401 iamrole.go:89] ListRoles: { } I0803 22:18:20.148958 20401 executor.go:111] Tasks: 0 done / 116 total; 47 can run I0803 22:18:20.149330 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0803 22:18:20.149517 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0803 22:18:20.149568 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:20.149609 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0803 22:18:20.149767 20401 api.go:53] PageSize not implemented I0803 22:18:20.149779 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.149902 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0803 22:18:20.149940 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0803 22:18:20.149973 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0803 22:18:20.150121 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } I0803 22:18:20.150316 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0803 22:18:20.150427 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.150581 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:20.151621 20401 executor.go:111] Tasks: 47 done / 116 total; 27 can run I0803 22:18:20.151778 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0803 22:18:20.151911 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0803 22:18:20.151918 20401 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0803 22:18:20.151948 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] }] } I0803 22:18:20.152040 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0803 22:18:20.152078 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.152238 20401 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0803 22:18:20.152309 20401 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0803 22:18:20.152341 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0803 22:18:20.152247 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0803 22:18:20.152481 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:20.152574 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.152748 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0803 22:18:20.152841 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:20.152965 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:20.154030 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0803 22:18:20.154391 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:20.154494 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.154651 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0803 22:18:20.154744 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.154905 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:20.155033 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:20.155153 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.155561 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0803 22:18:20.155682 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:18:20.160714 20401 executor.go:111] Tasks: 74 done / 116 total; 39 can run I0803 22:18:20.161061 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:20.161216 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0803 22:18:20.161339 20401 loadbalancers.go:34] PageSize not implemented I0803 22:18:20.161361 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:20.161422 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:20.161380 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.161476 20401 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0803 22:18:20.161568 20401 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0803 22:18:20.161609 20401 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0803 22:18:20.161619 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.161641 20401 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0803 22:18:20.161759 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:20.161828 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { Names: ["api-complex-example-com-vd3t5n"] } I0803 22:18:20.161865 20401 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0803 22:18:20.161896 20401 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/1" } I0803 22:18:20.161928 20401 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/2" } I0803 22:18:20.161962 20401 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0803 22:18:20.162054 20401 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0803 22:18:20.162134 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:20.162121 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:20.162173 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:20.162328 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.162437 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:20.162542 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:20.162634 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:20.162725 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:20.162803 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.165250 20401 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0803 22:18:20.165344 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:20.165446 20401 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0803 22:18:20.165479 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0803 22:18:20.165514 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:20.165594 20401 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0803 22:18:20.165637 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:20.165672 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:20.165722 20401 dns.go:238] Pre-creating DNS records I0803 22:18:20.174042 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:20.177927 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.178034 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.216597 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:20.318126 20401 iamrole.go:89] ListRoles: { } I0803 22:18:20.320659 20401 executor.go:111] Tasks: 0 done / 116 total; 47 can run I0803 22:18:20.321929 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:20.325249 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } W0803 22:18:20.327245 20401 api.go:53] PageSize not implemented I0803 22:18:20.327305 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.327491 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0803 22:18:20.327607 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.327748 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0803 22:18:20.327858 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0803 22:18:20.327963 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.328078 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0803 22:18:20.328113 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0803 22:18:20.328154 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0803 22:18:20.328370 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:20.329182 20401 executor.go:111] Tasks: 47 done / 116 total; 27 can run I0803 22:18:20.329336 20401 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0803 22:18:20.329337 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:20.329444 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0803 22:18:20.329475 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] }] } I0803 22:18:20.329529 20401 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0803 22:18:20.329684 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0803 22:18:20.329719 20401 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0803 22:18:20.329623 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.329805 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0803 22:18:20.329883 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:18:20.329972 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.330132 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0803 22:18:20.330183 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I0803 22:18:20.330374 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0803 22:18:20.330533 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:20.330603 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0803 22:18:20.330803 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0803 22:18:20.330878 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.331014 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0803 22:18:20.331086 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0803 22:18:20.331235 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.331435 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:20.331605 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:20.331756 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:20.331947 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:20.335365 20401 executor.go:111] Tasks: 74 done / 116 total; 39 can run I0803 22:18:20.335509 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:20.335618 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:20.335927 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0803 22:18:20.336019 20401 loadbalancers.go:34] PageSize not implemented I0803 22:18:20.335977 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:20.336037 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:20.336088 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:20.336128 20401 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0803 22:18:20.336171 20401 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0803 22:18:20.336206 20401 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0803 22:18:20.336237 20401 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0803 22:18:20.336263 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:20.336384 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.336597 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.336783 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.336862 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:20.336941 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:20.337033 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:20.337114 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.339389 20401 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0803 22:18:20.339643 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:20.339761 20401 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0803 22:18:20.339789 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0803 22:18:20.339819 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:20.339931 20401 executor.go:111] Tasks: 116 done / 116 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.additionalSecurityGroups=sg-123456] I0803 22:18:20.343653 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:20.347278 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.359507 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:20.363370 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.363476 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.402747 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:20.506854 20401 iamrole.go:89] ListRoles: { } I0803 22:18:20.509196 20401 executor.go:111] Tasks: 0 done / 117 total; 48 can run I0803 22:18:20.509541 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.509887 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0803 22:18:20.509903 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:20.510024 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } W0803 22:18:20.510164 20401 api.go:53] PageSize not implemented I0803 22:18:20.510254 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0803 22:18:20.510397 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0803 22:18:20.510502 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.510640 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0803 22:18:20.510677 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0803 22:18:20.510739 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:20.510786 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } I0803 22:18:20.510954 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0803 22:18:20.511066 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.512618 20401 executor.go:111] Tasks: 48 done / 117 total; 27 can run I0803 22:18:20.512851 20401 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0803 22:18:20.512826 20401 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0803 22:18:20.512957 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0803 22:18:20.513001 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0803 22:18:20.512826 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:20.513086 20401 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0803 22:18:20.513095 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:20.513193 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0803 22:18:20.513279 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0803 22:18:20.513589 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0803 22:18:20.513702 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.513929 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I0803 22:18:20.514154 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] }] } I0803 22:18:20.514395 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0803 22:18:20.514464 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0803 22:18:20.514657 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.514830 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.514983 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0803 22:18:20.515166 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:20.515293 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:20.515410 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.515591 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:20.515704 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0803 22:18:20.515780 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:18:20.515854 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:20.523409 20401 executor.go:111] Tasks: 75 done / 117 total; 39 can run I0803 22:18:20.523644 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:20.523823 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0803 22:18:20.523946 20401 loadbalancers.go:34] PageSize not implemented I0803 22:18:20.523972 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:20.523952 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:20.524034 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:20.524089 20401 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0803 22:18:20.524145 20401 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0803 22:18:20.524200 20401 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0803 22:18:20.524245 20401 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0803 22:18:20.524213 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:20.524673 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.524897 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:20.525230 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:20.525464 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:20.525605 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:20.525772 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.525961 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.526168 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.528863 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.528944 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.529156 20401 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0803 22:18:20.529308 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:20.529443 20401 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0803 22:18:20.529478 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0803 22:18:20.529522 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:20.529643 20401 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0803 22:18:20.529706 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:20.529763 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:20.529818 20401 dns.go:238] Pre-creating DNS records I0803 22:18:20.538821 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:20.542693 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.542796 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.581056 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:20.679004 20401 iamrole.go:89] ListRoles: { } I0803 22:18:20.680951 20401 executor.go:111] Tasks: 0 done / 117 total; 48 can run I0803 22:18:20.681104 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0803 22:18:20.681185 20401 api.go:53] PageSize not implemented I0803 22:18:20.681208 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0803 22:18:20.681232 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0803 22:18:20.681277 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0803 22:18:20.682765 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.683009 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:20.683105 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0803 22:18:20.683263 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0803 22:18:20.683401 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0803 22:18:20.683413 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:20.684007 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.684222 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0803 22:18:20.684692 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.689087 20401 executor.go:111] Tasks: 48 done / 117 total; 27 can run I0803 22:18:20.689229 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0803 22:18:20.689231 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.689373 20401 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0803 22:18:20.689388 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.689463 20401 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0803 22:18:20.689553 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0803 22:18:20.689553 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:20.689613 20401 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0803 22:18:20.689742 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0803 22:18:20.689681 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0803 22:18:20.689864 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0803 22:18:20.690089 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0803 22:18:20.690158 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0803 22:18:20.690407 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:20.690503 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.690719 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:20.690829 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.690969 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0803 22:18:20.691120 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:20.691170 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.691305 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:20.691397 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:18:20.691440 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:20.691524 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.691653 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0803 22:18:20.695089 20401 executor.go:111] Tasks: 75 done / 117 total; 39 can run I0803 22:18:20.695306 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.695510 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:20.695611 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0803 22:18:20.695705 20401 loadbalancers.go:34] PageSize not implemented I0803 22:18:20.695648 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.695724 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:20.695865 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.695889 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:20.695934 20401 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0803 22:18:20.695975 20401 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0803 22:18:20.696009 20401 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0803 22:18:20.696039 20401 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0803 22:18:20.696131 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:20.696283 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:20.696408 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:20.696621 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:20.696787 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:20.697135 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:20.697428 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.700034 20401 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0803 22:18:20.700137 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:20.700244 20401 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0803 22:18:20.700271 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0803 22:18:20.700296 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:20.700450 20401 executor.go:111] Tasks: 117 done / 117 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t3.large] I0803 22:18:20.705560 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:20.709022 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.709123 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.718664 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:20.722565 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.722664 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.722723 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.762051 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:20.860488 20401 iamrole.go:89] ListRoles: { } I0803 22:18:20.862838 20401 executor.go:111] Tasks: 0 done / 117 total; 48 can run I0803 22:18:20.863206 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } W0803 22:18:20.863345 20401 api.go:53] PageSize not implemented I0803 22:18:20.863375 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.863523 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:20.863554 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.863708 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.863817 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0803 22:18:20.863910 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0803 22:18:20.863974 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0803 22:18:20.864038 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0803 22:18:20.864101 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0803 22:18:20.864180 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0803 22:18:20.864204 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0803 22:18:20.864246 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:20.866215 20401 executor.go:111] Tasks: 48 done / 117 total; 27 can run I0803 22:18:20.866499 20401 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0803 22:18:20.866369 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0803 22:18:20.866563 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0803 22:18:20.866513 20401 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0803 22:18:20.866583 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.866647 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0803 22:18:20.866777 20401 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0803 22:18:20.866758 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0803 22:18:20.866855 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0803 22:18:20.866933 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0803 22:18:20.866989 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.867114 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0803 22:18:20.867161 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:20.867237 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:20.867310 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.867425 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0803 22:18:20.867461 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:20.867510 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.867619 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I0803 22:18:20.867721 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:20.867791 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:20.867856 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0803 22:18:20.867970 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:20.868033 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:20.868230 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:18:20.871752 20401 executor.go:111] Tasks: 75 done / 117 total; 39 can run I0803 22:18:20.871877 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:20.871985 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:20.872025 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0803 22:18:20.872149 20401 loadbalancers.go:34] PageSize not implemented I0803 22:18:20.872169 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:20.872165 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:20.872228 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:20.872270 20401 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0803 22:18:20.872264 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:20.872315 20401 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0803 22:18:20.872351 20401 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0803 22:18:20.872393 20401 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0803 22:18:20.872565 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.872741 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.872902 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:20.873255 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:20.873625 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.873797 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.873868 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:20.876257 20401 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0803 22:18:20.876357 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:20.876444 20401 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0803 22:18:20.876469 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0803 22:18:20.876493 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:20.876716 20401 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0803 22:18:20.876772 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:20.876808 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:20.876869 20401 dns.go:238] Pre-creating DNS records I0803 22:18:20.885045 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:20.888959 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.889067 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.889133 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:20.927620 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:21.031002 20401 iamrole.go:89] ListRoles: { } I0803 22:18:21.033256 20401 executor.go:111] Tasks: 0 done / 117 total; 48 can run I0803 22:18:21.033494 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.033708 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0803 22:18:21.033835 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0803 22:18:21.033992 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } W0803 22:18:21.034118 20401 api.go:53] PageSize not implemented I0803 22:18:21.034161 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:21.034225 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.034497 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.034673 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0803 22:18:21.034725 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0803 22:18:21.034822 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:21.034908 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.035132 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0803 22:18:21.035309 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0803 22:18:21.036516 20401 executor.go:111] Tasks: 48 done / 117 total; 27 can run I0803 22:18:21.036652 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.036715 20401 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0803 22:18:21.036811 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0803 22:18:21.036811 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:21.036998 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0803 22:18:21.036948 20401 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0803 22:18:21.037058 20401 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0803 22:18:21.037057 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0803 22:18:21.036949 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0803 22:18:21.037230 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:21.037299 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.037443 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0803 22:18:21.038476 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.039019 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0803 22:18:21.039128 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:21.039304 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.039517 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0803 22:18:21.039793 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:21.039969 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.040182 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I0803 22:18:21.040388 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:21.040545 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0803 22:18:21.040805 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:21.040969 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:18:21.041118 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0803 22:18:21.046712 20401 executor.go:111] Tasks: 75 done / 117 total; 39 can run I0803 22:18:21.047125 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.047171 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0803 22:18:21.047281 20401 loadbalancers.go:34] PageSize not implemented I0803 22:18:21.047308 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:21.047371 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:21.047371 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.047415 20401 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0803 22:18:21.047461 20401 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0803 22:18:21.047496 20401 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0803 22:18:21.047531 20401 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0803 22:18:21.047668 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:21.047878 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:21.048045 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:21.048240 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:21.048435 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:21.048552 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:21.048723 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.048877 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:21.049005 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.050095 20401 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0803 22:18:21.050218 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:21.050306 20401 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0803 22:18:21.050333 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0803 22:18:21.050359 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:21.050443 20401 executor.go:111] Tasks: 117 done / 117 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t2.medium] I0803 22:18:21.053700 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:21.057658 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.057750 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.057980 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.067735 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:21.071597 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.071694 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.071755 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.071810 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.110327 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:21.215642 20401 iamrole.go:89] ListRoles: { } I0803 22:18:21.217923 20401 executor.go:111] Tasks: 0 done / 117 total; 48 can run I0803 22:18:21.218213 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } I0803 22:18:21.218362 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.218530 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:21.218585 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } W0803 22:18:21.218750 20401 api.go:53] PageSize not implemented I0803 22:18:21.218697 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0803 22:18:21.218934 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0803 22:18:21.219101 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0803 22:18:21.219249 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0803 22:18:21.219381 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.219540 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0803 22:18:21.219588 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0803 22:18:21.219638 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.219923 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:21.220761 20401 executor.go:111] Tasks: 48 done / 117 total; 27 can run I0803 22:18:21.220878 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] }] } I0803 22:18:21.220950 20401 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0803 22:18:21.221041 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0803 22:18:21.221019 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:21.221125 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0803 22:18:21.221126 20401 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0803 22:18:21.221195 20401 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0803 22:18:21.221213 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0803 22:18:21.221175 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:21.221260 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0803 22:18:21.221310 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.221468 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0803 22:18:21.221514 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.221651 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.221783 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I0803 22:18:21.221909 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0803 22:18:21.222073 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0803 22:18:21.222158 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.222299 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:18:21.222354 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:21.222517 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0803 22:18:21.222771 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:21.222856 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0803 22:18:21.223017 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:21.223145 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:21.226945 20401 executor.go:111] Tasks: 75 done / 117 total; 39 can run I0803 22:18:21.227067 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:21.227280 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.227429 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:21.227441 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0803 22:18:21.227626 20401 loadbalancers.go:34] PageSize not implemented I0803 22:18:21.227655 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:21.227729 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:21.227790 20401 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0803 22:18:21.227771 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:21.227852 20401 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0803 22:18:21.227910 20401 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0803 22:18:21.227967 20401 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0803 22:18:21.228209 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:21.228662 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:21.228798 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:21.228969 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.229098 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:21.229238 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.229360 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.232308 20401 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0803 22:18:21.232502 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:21.232603 20401 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0803 22:18:21.232629 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0803 22:18:21.232655 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:21.232736 20401 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0803 22:18:21.232779 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:21.232811 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:21.232861 20401 dns.go:238] Pre-creating DNS records I0803 22:18:21.240567 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:21.244423 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.244522 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.244583 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.244636 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.283260 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:21.382700 20401 iamrole.go:89] ListRoles: { } I0803 22:18:21.385004 20401 executor.go:111] Tasks: 0 done / 117 total; 48 can run I0803 22:18:21.385202 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.385416 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } W0803 22:18:21.385594 20401 api.go:53] PageSize not implemented I0803 22:18:21.385650 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0803 22:18:21.385833 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0803 22:18:21.385918 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:21.386007 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.386186 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.386324 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0803 22:18:21.386430 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0803 22:18:21.386529 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.386631 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0803 22:18:21.386665 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0803 22:18:21.386720 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:21.388272 20401 executor.go:111] Tasks: 48 done / 117 total; 27 can run I0803 22:18:21.388499 20401 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0803 22:18:21.388426 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.388574 20401 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0803 22:18:21.388628 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.388737 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0803 22:18:21.388687 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0803 22:18:21.388827 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:21.388917 20401 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0803 22:18:21.388960 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:21.389106 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0803 22:18:21.389187 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:18:21.389803 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0803 22:18:21.389002 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0803 22:18:21.390099 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:21.390295 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.390467 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:21.390554 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0803 22:18:21.390702 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0803 22:18:21.390937 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:21.391061 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.391254 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0803 22:18:21.391463 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.391696 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0803 22:18:21.392225 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0803 22:18:21.392326 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:21.398137 20401 executor.go:111] Tasks: 75 done / 117 total; 39 can run I0803 22:18:21.398399 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:21.398545 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:21.398621 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0803 22:18:21.398711 20401 loadbalancers.go:34] PageSize not implemented I0803 22:18:21.398732 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:21.398800 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:21.398842 20401 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0803 22:18:21.398882 20401 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0803 22:18:21.398866 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:21.398919 20401 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0803 22:18:21.399044 20401 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0803 22:18:21.399042 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.399216 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:21.399683 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:21.399887 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.400053 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:21.400230 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:21.400394 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.400587 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.403244 20401 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0803 22:18:21.403341 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:21.403455 20401 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0803 22:18:21.403513 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0803 22:18:21.403575 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:21.403762 20401 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0803 22:18:21.404072 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0803 22:18:21.404132 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0803 22:18:21.404170 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0803 22:18:21.404206 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0803 22:18:21.404247 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0803 22:18:21.404281 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0803 22:18:21.404318 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0803 22:18:21.404351 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0803 22:18:21.404391 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0803 22:18:21.404435 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0803 22:18:21.404482 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0803 22:18:21.404516 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0803 22:18:21.404552 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0803 22:18:21.404593 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0803 22:18:21.404628 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0803 22:18:21.404668 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0803 22:18:21.404709 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0803 22:18:21.404747 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0803 22:18:21.404786 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0803 22:18:21.404826 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0803 22:18:21.404861 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0803 22:18:21.404901 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0803 22:18:21.404940 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0803 22:18:21.404975 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0803 22:18:21.405009 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0803 22:18:21.405049 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0803 22:18:21.405088 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0803 22:18:21.405127 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0803 22:18:21.407936 20401 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0803 22:18:21.407969 20401 keypairs.go:95] DescribeKeyPairs: { } I0803 22:18:21.408006 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.408067 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0803 22:18:21.408136 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.408201 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.408251 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.408312 20401 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.408354 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.408401 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0803 22:18:21.408458 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.408531 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0803 22:18:21.408576 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } W0803 22:18:21.408633 20401 api.go:53] PageSize not implemented I0803 22:18:21.408652 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0803 22:18:21.408679 20401 loadbalancers.go:34] PageSize not implemented I0803 22:18:21.408694 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0803 22:18:21.408733 20401 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0803 22:18:21.408757 20401 targetgroups.go:35] PageSize not implemented I0803 22:18:21.408794 20401 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2" ] } I0803 22:18:21.408867 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:21.408896 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:21.408938 20401 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0803 22:18:21.408972 20401 iamrole.go:89] ListRoles: { } I0803 22:18:21.409015 20401 internetgateways.go:101] DescribeInternetGateways: { } I0803 22:18:21.409066 20401 routetable.go:53] DescribeRouteTables: { } I0803 22:18:21.409113 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0803 22:18:21.409679 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:master-us-test-1a.masters.complex.example.com ok I0803 22:18:21.409776 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } autoscaling-config:lt-1 ok I0803 22:18:21.409738 20401 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4 ok I0803 22:18:21.409856 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } autoscaling-group:nodes.complex.example.com ok I0803 22:18:21.409857 20401 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0803 22:18:21.409926 20401 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5" } security-group:sg-1 ok I0803 22:18:21.409979 20401 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5 ok iam-instance-profile:masters.complex.example.com ok I0803 22:18:21.409997 20401 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1 ok I0803 22:18:21.410027 20401 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0803 22:18:21.409911 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.complex.example.com. ok I0803 22:18:21.410106 20401 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } iam-instance-profile:nodes.complex.example.com ok I0803 22:18:21.409957 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }, { FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 } ] } W0803 22:18:21.410426 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:21.410455 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0803 22:18:21.410503 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:21.410583 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 } ] } W0803 22:18:21.410825 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:21.410845 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0803 22:18:21.410898 20401 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0803 22:18:21.410965 20401 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-2 ok I0803 22:18:21.411050 20401 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I0803 22:18:21.411103 20401 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0803 22:18:21.411165 20401 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0803 22:18:21.411206 20401 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0803 22:18:21.411315 20401 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.complex.example.com" } I0803 22:18:21.411394 20401 iamrole.go:180] ListAttachedRolePolicies: nodes.complex.example.com I0803 22:18:21.411344 20401 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0803 22:18:21.411420 20401 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.complex.example.com" } I0803 22:18:21.411498 20401 iamrole.go:180] ListAttachedRolePolicies: masters.complex.example.com I0803 22:18:21.411519 20401 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0803 22:18:21.411444 20401 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0803 22:18:21.411560 20401 iamrole.go:137] DeleteRole: { RoleName: "masters.complex.example.com" } iam-role:masters.complex.example.com ok I0803 22:18:21.411591 20401 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0803 22:18:21.411629 20401 iamrole.go:137] DeleteRole: { RoleName: "nodes.complex.example.com" } iam-role:nodes.complex.example.com ok I0803 22:18:21.411701 20401 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0803 22:18:21.411786 20401 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0803 22:18:21.411865 20401 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0803 22:18:21.412168 20401 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleComplex (1.80s) === RUN TestLifecycleExternalLB I0803 22:18:21.412426 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:21.412682 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:21.412728 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:21.412768 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:21.412816 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:21.412859 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:21.412888 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:21.412922 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:21.412947 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:21.413000 20401 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster externallb.example.com I0803 22:18:21.437499 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:21.440389 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.440487 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.479151 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:21.576819 20401 iamrole.go:89] ListRoles: { } I0803 22:18:21.578725 20401 executor.go:111] Tasks: 0 done / 92 total; 50 can run I0803 22:18:21.579027 20401 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } W0803 22:18:21.579197 20401 api.go:53] PageSize not implemented I0803 22:18:21.579205 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0803 22:18:21.579152 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } W0803 22:18:21.579397 20401 api.go:53] PageSize not implemented W0803 22:18:21.579453 20401 api.go:53] PageSize not implemented I0803 22:18:21.579385 20401 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0803 22:18:21.579474 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } I0803 22:18:21.579459 20401 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0803 22:18:21.579555 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0803 22:18:21.579580 20401 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I0803 22:18:21.579567 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0803 22:18:21.579730 20401 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0803 22:18:21.579873 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0803 22:18:21.579927 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:21.579939 20401 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0803 22:18:21.579924 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } I0803 22:18:21.580087 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0803 22:18:21.580102 20401 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I0803 22:18:21.579826 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0803 22:18:21.580279 20401 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0803 22:18:21.580425 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0803 22:18:21.580496 20401 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0803 22:18:21.580536 20401 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0803 22:18:21.580568 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0803 22:18:21.580610 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:21.580648 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0803 22:18:21.580818 20401 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0803 22:18:21.580964 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0803 22:18:21.581006 20401 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0803 22:18:21.581124 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:21.581167 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:21.581325 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0803 22:18:21.581505 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0803 22:18:21.582205 20401 executor.go:111] Tasks: 50 done / 92 total; 19 can run I0803 22:18:21.582454 20401 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I0803 22:18:21.582403 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0803 22:18:21.582524 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0803 22:18:21.582631 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0803 22:18:21.582624 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0803 22:18:21.582753 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0803 22:18:21.582718 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] }], VpcId: "vpc-1" } I0803 22:18:21.582776 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0803 22:18:21.582871 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I0803 22:18:21.582830 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0803 22:18:21.582882 20401 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0803 22:18:21.583008 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0803 22:18:21.583080 20401 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" } ] } I0803 22:18:21.583180 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:21.583325 20401 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0803 22:18:21.583383 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0803 22:18:21.583463 20401 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0803 22:18:21.583575 20401 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0803 22:18:21.583690 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0803 22:18:21.583753 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:21.583954 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0803 22:18:21.584039 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:18:21.584086 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0803 22:18:21.584185 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0803 22:18:21.584240 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:21.584282 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.externallb.example.com" }] }], VpcId: "vpc-1" } I0803 22:18:21.584370 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0803 22:18:21.584417 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:21.584475 20401 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0803 22:18:21.584543 20401 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0803 22:18:21.587348 20401 executor.go:111] Tasks: 69 done / 92 total; 21 can run I0803 22:18:21.587552 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" }] }] } I0803 22:18:21.587799 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0803 22:18:21.588045 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:21.588234 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0803 22:18:21.588286 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0803 22:18:21.588419 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.588536 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.588721 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-1to2379-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0803 22:18:21.588952 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:21.589043 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:21.589115 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0803 22:18:21.589269 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0803 22:18:21.589415 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0803 22:18:21.589557 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0803 22:18:21.589692 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-udp-1to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0803 22:18:21.589829 20401 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0803 22:18:21.589888 20401 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0803 22:18:21.589936 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-2382to4000-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0803 22:18:21.590866 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0803 22:18:21.591405 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0803 22:18:21.591666 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0803 22:18:21.591933 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0803 22:18:21.592214 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-4003to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0803 22:18:21.592421 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.592554 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.595628 20401 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0803 22:18:21.595913 20401 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0803 22:18:21.595971 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:21.596011 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:21.596074 20401 dns.go:238] Pre-creating DNS records I0803 22:18:21.596110 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:21.602088 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:21.605378 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.605523 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.644635 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:21.739685 20401 iamrole.go:89] ListRoles: { } I0803 22:18:21.741232 20401 executor.go:111] Tasks: 0 done / 92 total; 50 can run I0803 22:18:21.741510 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } W0803 22:18:21.741679 20401 api.go:53] PageSize not implemented I0803 22:18:21.741700 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0803 22:18:21.741761 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0803 22:18:21.741724 20401 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0803 22:18:21.741849 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0803 22:18:21.741819 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } W0803 22:18:21.741962 20401 api.go:53] PageSize not implemented I0803 22:18:21.741940 20401 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0803 22:18:21.741989 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0803 22:18:21.742138 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } W0803 22:18:21.743402 20401 api.go:53] PageSize not implemented I0803 22:18:21.743343 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] }] } I0803 22:18:21.741915 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:21.744171 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:21.742054 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0803 22:18:21.744344 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:21.744439 20401 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0803 22:18:21.744549 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0803 22:18:21.744659 20401 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0803 22:18:21.744820 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0803 22:18:21.749593 20401 executor.go:111] Tasks: 50 done / 92 total; 19 can run I0803 22:18:21.749810 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:21.749961 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0803 22:18:21.750109 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I0803 22:18:21.750172 20401 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I0803 22:18:21.750100 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0803 22:18:21.750273 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:18:21.750324 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:21.750420 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0803 22:18:21.750593 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:21.750650 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0803 22:18:21.750780 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0803 22:18:21.750912 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:21.755104 20401 executor.go:111] Tasks: 69 done / 92 total; 21 can run I0803 22:18:21.755273 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:21.755402 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:21.755566 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:21.755867 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.756026 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.756233 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.756524 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.759125 20401 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0803 22:18:21.759406 20401 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0803 22:18:21.762617 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0803 22:18:21.762684 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0803 22:18:21.762722 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0803 22:18:21.762756 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0803 22:18:21.762796 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0803 22:18:21.762831 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0803 22:18:21.762863 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0803 22:18:21.762898 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0803 22:18:21.762935 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0803 22:18:21.762970 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0803 22:18:21.763017 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0803 22:18:21.763057 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0803 22:18:21.763089 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0803 22:18:21.763124 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0803 22:18:21.763162 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0803 22:18:21.763196 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0803 22:18:21.763232 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0803 22:18:21.763270 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0803 22:18:21.763309 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0803 22:18:21.763345 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0803 22:18:21.763378 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0803 22:18:21.763412 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0803 22:18:21.763449 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0803 22:18:21.763485 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0803 22:18:21.763523 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0803 22:18:21.764890 20401 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0803 22:18:21.764919 20401 keypairs.go:95] DescribeKeyPairs: { } I0803 22:18:21.764959 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0803 22:18:21.765013 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0803 22:18:21.765071 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0803 22:18:21.765131 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0803 22:18:21.765179 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0803 22:18:21.765238 20401 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0803 22:18:21.765282 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0803 22:18:21.765323 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0803 22:18:21.765373 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0803 22:18:21.765427 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0803 22:18:21.765493 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } W0803 22:18:21.765552 20401 api.go:53] PageSize not implemented I0803 22:18:21.765575 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0803 22:18:21.765603 20401 loadbalancers.go:34] PageSize not implemented I0803 22:18:21.765622 20401 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0803 22:18:21.765647 20401 targetgroups.go:35] PageSize not implemented I0803 22:18:21.765663 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0803 22:18:21.765712 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:21.765741 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:21.765791 20401 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0803 22:18:21.765827 20401 iamrole.go:89] ListRoles: { } I0803 22:18:21.765867 20401 internetgateways.go:101] DescribeInternetGateways: { } I0803 22:18:21.765910 20401 routetable.go:53] DescribeRouteTables: { } I0803 22:18:21.765943 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:21.766391 20401 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } autoscaling-group:master-us-test-1a.masters.externallb.example.com ok volume:vol-1 ok I0803 22:18:21.766458 20401 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } autoscaling-group:nodes.externallb.example.com ok volume:vol-2 ok autoscaling-config:lt-2 ok I0803 22:18:21.766566 20401 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0803 22:18:21.766647 20401 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } autoscaling-config:lt-1 ok I0803 22:18:21.766514 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.externallb.example.com. ok I0803 22:18:21.766707 20401 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } iam-instance-profile:masters.externallb.example.com ok I0803 22:18:21.766750 20401 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } iam-instance-profile:nodes.externallb.example.com ok I0803 22:18:21.766812 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:21.766869 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0803 22:18:21.767106 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:21.767139 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0803 22:18:21.767178 20401 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0803 22:18:21.767246 20401 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0803 22:18:21.767282 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:21.767335 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0803 22:18:21.767447 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:21.767462 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0803 22:18:21.767501 20401 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0803 22:18:21.767603 20401 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.externallb.example.com" } I0803 22:18:21.767652 20401 iamrole.go:180] ListAttachedRolePolicies: nodes.externallb.example.com I0803 22:18:21.767670 20401 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0803 22:18:21.767703 20401 iamrole.go:137] DeleteRole: { RoleName: "nodes.externallb.example.com" } iam-role:nodes.externallb.example.com ok I0803 22:18:21.767677 20401 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0803 22:18:21.767735 20401 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.externallb.example.com" } I0803 22:18:21.767776 20401 iamrole.go:180] ListAttachedRolePolicies: masters.externallb.example.com I0803 22:18:21.767789 20401 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0803 22:18:21.767816 20401 iamrole.go:137] DeleteRole: { RoleName: "masters.externallb.example.com" } iam-role:masters.externallb.example.com ok I0803 22:18:21.767884 20401 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0803 22:18:21.767967 20401 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0803 22:18:21.768046 20401 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0803 22:18:21.768316 20401 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleExternalLB (0.36s) === RUN TestLifecycleSharedSubnet I0803 22:18:21.768589 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:21.768861 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:21.768908 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:21.768949 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:21.768995 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:21.769036 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:21.769067 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:21.769102 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:21.769129 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:21.769181 20401 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedsubnet.example.com I0803 22:18:21.793336 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:21.796192 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.796284 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.834640 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:21.936198 20401 iamrole.go:89] ListRoles: { } I0803 22:18:21.937623 20401 executor.go:111] Tasks: 0 done / 78 total; 42 can run I0803 22:18:21.937888 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:21.937943 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:21.937960 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0803 22:18:21.938116 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0803 22:18:21.938182 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0803 22:18:21.938235 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0803 22:18:21.938465 20401 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0803 22:18:21.938240 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } I0803 22:18:21.938692 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0803 22:18:21.938799 20401 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0803 22:18:21.938837 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:21.939049 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } I0803 22:18:21.939136 20401 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "masters.sharedsubnet.example.com" }] } I0803 22:18:21.939076 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0803 22:18:21.939300 20401 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0803 22:18:21.939463 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0803 22:18:21.939517 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0803 22:18:21.940539 20401 executor.go:111] Tasks: 42 done / 78 total; 16 can run I0803 22:18:21.940677 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0803 22:18:21.940819 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0803 22:18:21.940973 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0803 22:18:21.941006 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0803 22:18:21.941140 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0803 22:18:21.941183 20401 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0803 22:18:21.941202 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0803 22:18:21.941270 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0803 22:18:21.941344 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0803 22:18:21.941411 20401 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] } I0803 22:18:21.941370 20401 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0803 22:18:21.941505 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.sharedsubnet.example.com" }] }], VpcId: "vpc-12345678" } I0803 22:18:21.941648 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0803 22:18:21.941714 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:21.941760 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }], VpcId: "vpc-12345678" } I0803 22:18:21.941845 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0803 22:18:21.941896 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:21.942830 20401 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0803 22:18:21.946206 20401 executor.go:111] Tasks: 58 done / 78 total; 18 can run I0803 22:18:21.946410 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-1to2379-masters.sharedsubnet.example.com" }] }] } I0803 22:18:21.946602 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0803 22:18:21.946820 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0803 22:18:21.947040 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0803 22:18:21.947257 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0803 22:18:21.947473 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" }] }] } I0803 22:18:21.947718 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.947815 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0803 22:18:21.948025 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-4003to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0803 22:18:21.948177 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0803 22:18:21.948320 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.948434 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.948524 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-2382to4000-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0803 22:18:21.948741 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0803 22:18:21.948909 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0803 22:18:21.949052 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.950000 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0803 22:18:21.950514 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-udp-1to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0803 22:18:21.956624 20401 executor.go:111] Tasks: 76 done / 78 total; 2 can run I0803 22:18:21.956938 20401 executor.go:111] Tasks: 78 done / 78 total; 0 can run I0803 22:18:21.956993 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:21.957030 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:21.957091 20401 dns.go:238] Pre-creating DNS records I0803 22:18:21.957122 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:21.963240 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:21.966085 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:21.966182 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:22.005175 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:22.103301 20401 iamrole.go:89] ListRoles: { } I0803 22:18:22.104915 20401 executor.go:111] Tasks: 0 done / 78 total; 42 can run I0803 22:18:22.105327 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0803 22:18:22.105627 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:22.106596 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:22.107037 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0803 22:18:22.107245 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0803 22:18:22.107310 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0803 22:18:22.108143 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0803 22:18:22.108396 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:22.113342 20401 executor.go:111] Tasks: 42 done / 78 total; 16 can run I0803 22:18:22.113480 20401 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0803 22:18:22.113552 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0803 22:18:22.113696 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0803 22:18:22.113693 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0803 22:18:22.113849 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0803 22:18:22.114016 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:22.114080 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0803 22:18:22.114288 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:22.118808 20401 executor.go:111] Tasks: 58 done / 78 total; 18 can run I0803 22:18:22.119028 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:22.119269 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:22.119556 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:22.119723 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:22.121502 20401 executor.go:111] Tasks: 76 done / 78 total; 2 can run I0803 22:18:22.121738 20401 executor.go:111] Tasks: 78 done / 78 total; 0 can run I0803 22:18:22.124857 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0803 22:18:22.124925 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0803 22:18:22.124961 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0803 22:18:22.124994 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0803 22:18:22.125030 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0803 22:18:22.125073 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0803 22:18:22.125110 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0803 22:18:22.125147 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0803 22:18:22.125188 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0803 22:18:22.125234 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0803 22:18:22.125272 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0803 22:18:22.125307 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0803 22:18:22.125340 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0803 22:18:22.125387 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0803 22:18:22.125443 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0803 22:18:22.125488 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0803 22:18:22.125524 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0803 22:18:22.125560 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0803 22:18:22.125599 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0803 22:18:22.125637 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0803 22:18:22.127056 20401 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0803 22:18:22.127089 20401 keypairs.go:95] DescribeKeyPairs: { } I0803 22:18:22.127130 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0803 22:18:22.127182 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0803 22:18:22.127240 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0803 22:18:22.127301 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0803 22:18:22.127344 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0803 22:18:22.127389 20401 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0803 22:18:22.127431 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0803 22:18:22.127469 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0803 22:18:22.127511 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0803 22:18:22.127555 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0803 22:18:22.127592 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } W0803 22:18:22.127643 20401 api.go:53] PageSize not implemented I0803 22:18:22.127667 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0803 22:18:22.127697 20401 loadbalancers.go:34] PageSize not implemented I0803 22:18:22.127717 20401 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0803 22:18:22.127743 20401 targetgroups.go:35] PageSize not implemented I0803 22:18:22.127762 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0803 22:18:22.127814 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:22.127845 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:22.127896 20401 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0803 22:18:22.127935 20401 iamrole.go:89] ListRoles: { } I0803 22:18:22.127983 20401 internetgateways.go:101] DescribeInternetGateways: { } I0803 22:18:22.128023 20401 routetable.go:53] DescribeRouteTables: { } I0803 22:18:22.128797 20401 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0803 22:18:22.128869 20401 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } autoscaling-group:nodes.sharedsubnet.example.com ok I0803 22:18:22.128847 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } iam-instance-profile:nodes.sharedsubnet.example.com ok autoscaling-config:lt-1 ok I0803 22:18:22.128950 20401 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0803 22:18:22.129035 20401 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } iam-instance-profile:masters.sharedsubnet.example.com ok autoscaling-group:master-us-test-1a.masters.sharedsubnet.example.com ok I0803 22:18:22.128990 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0803 22:18:22.129160 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions autoscaling-config:lt-2 ok I0803 22:18:22.129048 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:22.129208 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } route53-record:Z1AFAKE1ZON3YO/A/api.sharedsubnet.example.com. ok security-group:sg-1 ok I0803 22:18:22.129267 20401 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0803 22:18:22.129331 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:22.129388 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0803 22:18:22.129616 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:22.129637 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0803 22:18:22.129678 20401 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0803 22:18:22.129734 20401 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0803 22:18:22.129829 20401 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedsubnet.example.com" } I0803 22:18:22.129881 20401 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0803 22:18:22.129899 20401 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0803 22:18:22.129934 20401 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedsubnet.example.com" } iam-role:nodes.sharedsubnet.example.com ok I0803 22:18:22.129972 20401 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedsubnet.example.com" } I0803 22:18:22.130049 20401 iamrole.go:180] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0803 22:18:22.130065 20401 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0803 22:18:22.130102 20401 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedsubnet.example.com" } iam-role:masters.sharedsubnet.example.com ok I0803 22:18:22.130375 20401 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedSubnet (0.36s) === RUN TestLifecyclePrivateSharedSubnet I0803 22:18:22.130669 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:22.130939 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:22.130991 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:22.131034 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:22.131086 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:22.131129 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:22.131160 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:22.131196 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:22.131225 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:22.131282 20401 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-subnet.example.com I0803 22:18:22.159904 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:22.162958 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.163053 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.163121 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.203826 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:22.309572 20401 iamrole.go:89] ListRoles: { } I0803 22:18:22.311585 20401 executor.go:111] Tasks: 0 done / 104 total; 45 can run I0803 22:18:22.311969 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } I0803 22:18:22.312122 20401 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0803 22:18:22.312454 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:22.312324 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] }] } I0803 22:18:22.312656 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0803 22:18:22.312825 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0803 22:18:22.312722 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } I0803 22:18:22.312889 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0803 22:18:22.312921 20401 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0803 22:18:22.313058 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } I0803 22:18:22.312965 20401 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0803 22:18:22.313144 20401 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] } I0803 22:18:22.313191 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0803 22:18:22.313269 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] }] } I0803 22:18:22.313507 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:22.313608 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:22.313727 20401 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0803 22:18:22.313923 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0803 22:18:22.313983 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0803 22:18:22.314553 20401 executor.go:111] Tasks: 45 done / 104 total; 24 can run I0803 22:18:22.314723 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0803 22:18:22.314770 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0803 22:18:22.314878 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0803 22:18:22.314940 20401 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0803 22:18:22.314870 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I0803 22:18:22.315032 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0803 22:18:22.315063 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0803 22:18:22.315160 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0803 22:18:22.315230 20401 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0803 22:18:22.315151 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0803 22:18:22.315445 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0803 22:18:22.315569 20401 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0803 22:18:22.315589 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0803 22:18:22.315697 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:22.315750 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0803 22:18:22.315851 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0803 22:18:22.315901 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:22.315852 20401 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0803 22:18:22.315953 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0803 22:18:22.316213 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0803 22:18:22.316276 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0803 22:18:22.316435 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0803 22:18:22.316575 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0803 22:18:22.316680 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "bastion-elb.private-shared-subnet.example.com" }] }], VpcId: "vpc-12345678" } I0803 22:18:22.316808 20401 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" },{ Key: "SubnetType", Value: "Private" },{ Key: "kubernetes.io/role/internal-elb", Value: "1" }] } I0803 22:18:22.316905 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }], VpcId: "vpc-12345678" } I0803 22:18:22.317028 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0803 22:18:22.317107 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0803 22:18:22.317170 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0803 22:18:22.317285 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0803 22:18:22.317366 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0803 22:18:22.317455 20401 tags.go:42] CreateTags { Resources: ["subnet-abcdef"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" },{ Key: "SubnetType", Value: "Utility" },{ Key: "kubernetes.io/role/elb", Value: "1" }] } I0803 22:18:22.317543 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0803 22:18:22.317604 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0803 22:18:22.317648 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0803 22:18:22.317535 20401 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0803 22:18:22.319971 20401 executor.go:111] Tasks: 69 done / 104 total; 31 can run I0803 22:18:22.320200 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } W0803 22:18:22.320366 20401 api.go:53] PageSize not implemented I0803 22:18:22.320454 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0803 22:18:22.320525 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0803 22:18:22.320656 20401 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"], Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0803 22:18:22.320726 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0803 22:18:22.320712 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0803 22:18:22.321348 20401 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0803 22:18:22.321425 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-1to2379-masters.private-shared-subnet.example.com" }] }] } I0803 22:18:22.321469 20401 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } W0803 22:18:22.321809 20401 api.go:53] PageSize not implemented I0803 22:18:22.321737 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-subnet.example.com" }] }] } I0803 22:18:22.321854 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0803 22:18:22.321945 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-ingress-tcp-22to22-bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0803 22:18:22.322307 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0803 22:18:22.322214 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0803 22:18:22.322415 20401 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"], Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "api.private-shared-subnet.example.com" }] } I0803 22:18:22.322561 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0803 22:18:22.322638 20401 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0803 22:18:22.322490 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0803 22:18:22.322770 20401 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0803 22:18:22.322825 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.323006 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-4003to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0803 22:18:22.323335 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0803 22:18:22.323538 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0803 22:18:22.323834 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } I0803 22:18:22.324123 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.325122 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.325214 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0803 22:18:22.325455 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I0803 22:18:22.325600 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0803 22:18:22.325753 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0803 22:18:22.326646 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0803 22:18:22.326928 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.327106 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.327233 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-masters.private-shared-subnet.example.com" }] }] } I0803 22:18:22.327578 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0803 22:18:22.327755 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-2382to4000-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0803 22:18:22.327903 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0803 22:18:22.328041 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-udp-1to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0803 22:18:22.328185 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.328358 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" }] }] } I0803 22:18:22.328583 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0803 22:18:22.330801 20401 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0803 22:18:22.330926 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:22.331083 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:22.331214 20401 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0803 22:18:22.331268 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:22.331308 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:22.331369 20401 dns.go:238] Pre-creating DNS records I0803 22:18:22.331396 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:22.338076 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:22.341156 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.341254 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.341320 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.380633 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:22.484070 20401 iamrole.go:89] ListRoles: { } I0803 22:18:22.486069 20401 executor.go:111] Tasks: 0 done / 104 total; 45 can run I0803 22:18:22.486390 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:22.486468 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0803 22:18:22.486720 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:22.486842 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:22.487096 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0803 22:18:22.487229 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0803 22:18:22.487263 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0803 22:18:22.487321 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0803 22:18:22.489592 20401 executor.go:111] Tasks: 45 done / 104 total; 24 can run I0803 22:18:22.489820 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0803 22:18:22.489975 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0803 22:18:22.489914 20401 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0803 22:18:22.490158 20401 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0803 22:18:22.490104 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0803 22:18:22.490376 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:22.490455 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0803 22:18:22.490474 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I0803 22:18:22.490720 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0803 22:18:22.490812 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0803 22:18:22.491067 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0803 22:18:22.491133 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0803 22:18:22.491342 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0803 22:18:22.491484 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0803 22:18:22.491709 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0803 22:18:22.491826 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:22.495103 20401 executor.go:111] Tasks: 69 done / 104 total; 31 can run I0803 22:18:22.495251 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } W0803 22:18:22.495480 20401 api.go:53] PageSize not implemented I0803 22:18:22.495484 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.495516 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0803 22:18:22.495639 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0803 22:18:22.495657 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.495688 20401 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } W0803 22:18:22.495775 20401 api.go:53] PageSize not implemented I0803 22:18:22.495805 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0803 22:18:22.495900 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0803 22:18:22.495941 20401 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0803 22:18:22.495883 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.496104 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.496435 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.502485 20401 executor.go:111] Tasks: 100 done / 104 total; 4 can run W0803 22:18:22.502622 20401 api.go:53] PageSize not implemented W0803 22:18:22.502653 20401 api.go:53] PageSize not implemented I0803 22:18:22.502697 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:22.502704 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0803 22:18:22.502794 20401 dnsname.go:121] AliasTarget for "api.private-shared-subnet.example.com" is "api-private-shared-subnet-n2f8ak.elb.cloudmock.com" I0803 22:18:22.502822 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0803 22:18:22.502864 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0803 22:18:22.503432 20401 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0803 22:18:22.507029 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0803 22:18:22.507092 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0803 22:18:22.507128 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0803 22:18:22.507163 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0803 22:18:22.507201 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0803 22:18:22.507235 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0803 22:18:22.507271 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0803 22:18:22.507310 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0803 22:18:22.507348 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0803 22:18:22.507386 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0803 22:18:22.507427 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0803 22:18:22.507459 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0803 22:18:22.507491 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0803 22:18:22.507521 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0803 22:18:22.507555 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0803 22:18:22.507591 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0803 22:18:22.507627 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0803 22:18:22.507663 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0803 22:18:22.507698 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0803 22:18:22.507733 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0803 22:18:22.507771 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0803 22:18:22.507806 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0803 22:18:22.507843 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0803 22:18:22.507882 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0803 22:18:22.509506 20401 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0803 22:18:22.509536 20401 keypairs.go:95] DescribeKeyPairs: { } I0803 22:18:22.509577 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0803 22:18:22.509643 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0803 22:18:22.509723 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0803 22:18:22.509783 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0803 22:18:22.509826 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0803 22:18:22.509870 20401 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0803 22:18:22.509912 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0803 22:18:22.509950 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0803 22:18:22.509988 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0803 22:18:22.510034 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0803 22:18:22.510073 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } W0803 22:18:22.510118 20401 api.go:53] PageSize not implemented I0803 22:18:22.510136 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0803 22:18:22.510191 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0803 22:18:22.510219 20401 loadbalancers.go:34] PageSize not implemented I0803 22:18:22.510240 20401 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0803 22:18:22.510265 20401 targetgroups.go:35] PageSize not implemented I0803 22:18:22.510284 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0803 22:18:22.510338 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:22.510367 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:22.510420 20401 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0803 22:18:22.510459 20401 iamrole.go:89] ListRoles: { } I0803 22:18:22.510506 20401 internetgateways.go:101] DescribeInternetGateways: { } I0803 22:18:22.510548 20401 routetable.go:53] DescribeRouteTables: { } autoscaling-group:nodes.private-shared-subnet.example.com ok I0803 22:18:22.511072 20401 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } load-balancer:api-private-shared-subnet-n2f8ak ok autoscaling-group:master-us-test-1a.masters.private-shared-subnet.example.com ok I0803 22:18:22.511147 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0803 22:18:22.511230 20401 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } autoscaling-group:bastion.private-shared-subnet.example.com ok I0803 22:18:22.511316 20401 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } iam-instance-profile:masters.private-shared-subnet.example.com ok I0803 22:18:22.511301 20401 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } load-balancer:bastion-private-shared-su-5ol32q ok I0803 22:18:22.511136 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.private-shared-subnet.example.com ok I0803 22:18:22.511208 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0803 22:18:22.511421 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:22.511366 20401 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0803 22:18:22.511439 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0803 22:18:22.511451 20401 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } iam-instance-profile:bastions.private-shared-subnet.example.com ok I0803 22:18:22.511477 20401 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } I0803 22:18:22.511488 20401 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } keypair:key-1 ok I0803 22:18:22.511536 20401 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } iam-instance-profile:nodes.private-shared-subnet.example.com ok I0803 22:18:22.511548 20401 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-3 ok autoscaling-config:lt-1 ok I0803 22:18:22.511655 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:22.511712 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W0803 22:18:22.511829 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:22.511867 20401 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0803 22:18:22.511957 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0803 22:18:22.512029 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } W0803 22:18:22.512154 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:22.512177 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok autoscaling-config:lt-2 ok I0803 22:18:22.512234 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0803 22:18:22.512315 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0803 22:18:22.512333 20401 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-subnet.example.com" } I0803 22:18:22.512399 20401 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0803 22:18:22.512367 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0803 22:18:22.512448 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:22.512463 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0803 22:18:22.512422 20401 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-subnet.example.com" } I0803 22:18:22.512510 20401 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0803 22:18:22.512496 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:22.512527 20401 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0803 22:18:22.512561 20401 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-subnet.example.com" } iam-role:masters.private-shared-subnet.example.com ok I0803 22:18:22.512604 20401 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0803 22:18:22.512553 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } I0803 22:18:22.512685 20401 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-subnet.example.com" } W0803 22:18:22.512706 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions iam-role:nodes.private-shared-subnet.example.com ok I0803 22:18:22.512722 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0803 22:18:22.512730 20401 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-subnet.example.com" } I0803 22:18:22.512797 20401 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0803 22:18:22.512826 20401 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0803 22:18:22.512883 20401 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-subnet.example.com" } iam-role:bastions.private-shared-subnet.example.com ok I0803 22:18:22.513322 20401 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedSubnet (0.38s) === RUN TestLifecyclePrivateSharedIP I0803 22:18:22.513735 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:22.513979 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:22.514029 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:22.514073 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:22.514123 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:22.514188 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:22.514222 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:22.514264 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:22.514291 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:22.514346 20401 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-ip.example.com I0803 22:18:22.542326 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:22.545317 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.545411 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.545521 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.584501 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:22.683745 20401 iamrole.go:89] ListRoles: { } I0803 22:18:22.686042 20401 executor.go:111] Tasks: 0 done / 113 total; 45 can run I0803 22:18:22.686238 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } I0803 22:18:22.686255 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0803 22:18:22.686344 20401 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0803 22:18:22.686427 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0803 22:18:22.686506 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0803 22:18:22.686544 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0803 22:18:22.686503 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } I0803 22:18:22.686582 20401 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0803 22:18:22.686582 20401 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0803 22:18:22.686746 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0803 22:18:22.686790 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:22.686817 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0803 22:18:22.686941 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } I0803 22:18:22.687020 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:22.687093 20401 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0803 22:18:22.687139 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:22.687273 20401 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0803 22:18:22.687455 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0803 22:18:22.687516 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0803 22:18:22.689336 20401 executor.go:111] Tasks: 45 done / 113 total; 26 can run I0803 22:18:22.689583 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0803 22:18:22.689502 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0803 22:18:22.689665 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0803 22:18:22.689669 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0803 22:18:22.689830 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0803 22:18:22.689899 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0803 22:18:22.690035 20401 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0803 22:18:22.689983 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I0803 22:18:22.690085 20401 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0803 22:18:22.690178 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0803 22:18:22.690313 20401 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0803 22:18:22.690210 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I0803 22:18:22.690397 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0803 22:18:22.690458 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:22.690406 20401 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0803 22:18:22.690503 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0803 22:18:22.690721 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0803 22:18:22.690813 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0803 22:18:22.690861 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:22.690907 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I0803 22:18:22.691041 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0803 22:18:22.691101 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0803 22:18:22.691144 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0803 22:18:22.691258 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0803 22:18:22.691344 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0803 22:18:22.691392 20401 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-12345678" } I0803 22:18:22.691595 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0803 22:18:22.691690 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0803 22:18:22.691746 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0803 22:18:22.691791 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0803 22:18:22.692027 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" } ] }], VpcId: "vpc-12345678" } I0803 22:18:22.692063 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "bastion-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I0803 22:18:22.692214 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0803 22:18:22.692274 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0803 22:18:22.692317 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0803 22:18:22.691938 20401 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0803 22:18:22.692370 20401 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] } I0803 22:18:22.692462 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0803 22:18:22.692693 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] }], VpcId: "vpc-12345678" } I0803 22:18:22.692741 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0803 22:18:22.692871 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0803 22:18:22.692968 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0803 22:18:22.693028 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:18:22.693005 20401 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "private-shared-ip.example.com" } ] }], VpcId: "vpc-12345678" } I0803 22:18:22.693080 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0803 22:18:22.693133 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0803 22:18:22.693174 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0803 22:18:22.693219 20401 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] } I0803 22:18:22.694856 20401 executor.go:111] Tasks: 71 done / 113 total; 36 can run W0803 22:18:22.695118 20401 api.go:53] PageSize not implemented I0803 22:18:22.695057 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0803 22:18:22.695211 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0803 22:18:22.695257 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0803 22:18:22.695344 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:22.695364 20401 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"], Tags: [{ Key: "Name", Value: "api.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0803 22:18:22.695491 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0803 22:18:22.695529 20401 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-2" } I0803 22:18:22.695556 20401 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0803 22:18:22.695593 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:22.695636 20401 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0803 22:18:22.695720 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0803 22:18:22.695765 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } W0803 22:18:22.695741 20401 api.go:53] PageSize not implemented I0803 22:18:22.695811 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.695850 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0803 22:18:22.695927 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.696040 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0803 22:18:22.696081 20401 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"], Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0803 22:18:22.695992 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-ip.example.com" }] }] } I0803 22:18:22.696153 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0803 22:18:22.696233 20401 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0803 22:18:22.696286 20401 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0803 22:18:22.696315 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I0803 22:18:22.696476 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:22.696558 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:22.696638 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.696741 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0803 22:18:22.697018 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I0803 22:18:22.697161 20401 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0803 22:18:22.697258 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-1to2379-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0803 22:18:22.697422 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.697533 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.697658 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0803 22:18:22.697858 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0803 22:18:22.697983 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0803 22:18:22.698122 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0803 22:18:22.698178 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0803 22:18:22.698261 20401 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-2" } I0803 22:18:22.698356 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0803 22:18:22.698538 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0803 22:18:22.698693 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.698812 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0803 22:18:22.699007 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I0803 22:18:22.699256 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0803 22:18:22.699377 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-ingress-tcp-22to22-bastion.private-shared-ip.example.com" }] }] } I0803 22:18:22.699525 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0803 22:18:22.699666 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0803 22:18:22.699799 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0803 22:18:22.699848 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-4003to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0803 22:18:22.699988 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-2382to4000-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0803 22:18:22.700124 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-udp-1to65535-masters.private-shared-ip.example.com" }] }] } I0803 22:18:22.700262 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-ip.example.com" }] }] } I0803 22:18:22.700373 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } I0803 22:18:22.700474 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0803 22:18:22.700648 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0803 22:18:22.700699 20401 tags.go:42] CreateTags { Resources: ["eipalloc-12345678"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "shared" }] } I0803 22:18:22.700774 20401 executor.go:111] Tasks: 107 done / 113 total; 5 can run I0803 22:18:22.700957 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:22.700964 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:22.701039 20401 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-2"] }] } I0803 22:18:22.701132 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:22.701162 20401 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-12345678", SubnetId: "subnet-2", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0803 22:18:22.701255 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0803 22:18:22.701301 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0803 22:18:22.701344 20401 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0803 22:18:22.701411 20401 executor.go:111] Tasks: 112 done / 113 total; 1 can run I0803 22:18:22.701472 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:22.701596 20401 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I0803 22:18:22.701667 20401 executor.go:111] Tasks: 113 done / 113 total; 0 can run I0803 22:18:22.701716 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:22.701754 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:22.701807 20401 dns.go:238] Pre-creating DNS records I0803 22:18:22.701835 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:22.708027 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0803 22:18:22.711070 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.711165 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.711229 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.750661 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:22.851104 20401 iamrole.go:89] ListRoles: { } I0803 22:18:22.853283 20401 executor.go:111] Tasks: 0 done / 113 total; 45 can run I0803 22:18:22.853969 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0803 22:18:22.854120 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0803 22:18:22.854162 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0803 22:18:22.854147 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:22.854213 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0803 22:18:22.854425 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:22.854506 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:22.854623 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0803 22:18:22.856573 20401 executor.go:111] Tasks: 45 done / 113 total; 26 can run I0803 22:18:22.856780 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0803 22:18:22.856969 20401 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0803 22:18:22.856969 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0803 22:18:22.857082 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0803 22:18:22.857173 20401 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0803 22:18:22.857164 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:22.857299 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0803 22:18:22.857469 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0803 22:18:22.857702 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0803 22:18:22.857778 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0803 22:18:22.857977 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0803 22:18:22.858026 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0803 22:18:22.858162 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:18:22.858201 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0803 22:18:22.858332 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I0803 22:18:22.858482 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0803 22:18:22.858540 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0803 22:18:22.858743 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0803 22:18:22.858820 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0803 22:18:22.858978 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:22.861791 20401 executor.go:111] Tasks: 71 done / 113 total; 36 can run I0803 22:18:22.861961 20401 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } W0803 22:18:22.862062 20401 api.go:53] PageSize not implemented I0803 22:18:22.862053 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0803 22:18:22.862123 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0803 22:18:22.862256 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0803 22:18:22.862201 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.862322 20401 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } W0803 22:18:22.862400 20401 api.go:53] PageSize not implemented I0803 22:18:22.862347 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:22.862437 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0803 22:18:22.862601 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.862627 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0803 22:18:22.862976 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:22.863445 20401 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0803 22:18:22.864071 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.864297 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:22.864528 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0803 22:18:22.864658 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.864898 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.865304 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0803 22:18:22.869971 20401 executor.go:111] Tasks: 107 done / 113 total; 5 can run W0803 22:18:22.870106 20401 api.go:53] PageSize not implemented I0803 22:18:22.870136 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0803 22:18:22.870198 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0803 22:18:22.870309 20401 dnsname.go:121] AliasTarget for "api.private-shared-ip.example.com" is "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com" I0803 22:18:22.870262 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:22.870368 20401 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } W0803 22:18:22.870262 20401 api.go:53] PageSize not implemented I0803 22:18:22.870452 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0803 22:18:22.870649 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0803 22:18:22.870736 20401 executor.go:111] Tasks: 112 done / 113 total; 1 can run I0803 22:18:22.870807 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:22.870961 20401 executor.go:111] Tasks: 113 done / 113 total; 0 can run I0803 22:18:22.874410 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0803 22:18:22.874472 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0803 22:18:22.874512 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0803 22:18:22.874550 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0803 22:18:22.874590 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0803 22:18:22.874624 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0803 22:18:22.874659 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0803 22:18:22.874698 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0803 22:18:22.874738 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0803 22:18:22.874783 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0803 22:18:22.874826 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0803 22:18:22.874862 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0803 22:18:22.874896 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0803 22:18:22.874957 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0803 22:18:22.875021 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0803 22:18:22.875080 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0803 22:18:22.875136 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0803 22:18:22.875205 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0803 22:18:22.875271 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0803 22:18:22.875328 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0803 22:18:22.875369 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0803 22:18:22.875406 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0803 22:18:22.875466 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0803 22:18:22.875504 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0803 22:18:22.875543 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0803 22:18:22.875576 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0803 22:18:22.875609 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0803 22:18:22.875648 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0803 22:18:22.875688 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0803 22:18:22.877295 20401 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0803 22:18:22.877325 20401 keypairs.go:95] DescribeKeyPairs: { } I0803 22:18:22.877368 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0803 22:18:22.877434 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0803 22:18:22.877516 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0803 22:18:22.877579 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0803 22:18:22.877622 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0803 22:18:22.877666 20401 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0803 22:18:22.877707 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0803 22:18:22.877750 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0803 22:18:22.877803 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0803 22:18:22.877860 20401 routetable.go:53] DescribeRouteTables: { } I0803 22:18:22.877887 20401 natgateway.go:117] DescribeNatGateways: { } I0803 22:18:22.877929 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0803 22:18:22.877967 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } W0803 22:18:22.878014 20401 api.go:53] PageSize not implemented I0803 22:18:22.878029 20401 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I0803 22:18:22.878476 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0803 22:18:22.878529 20401 loadbalancers.go:34] PageSize not implemented I0803 22:18:22.878559 20401 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0803 22:18:22.878597 20401 targetgroups.go:35] PageSize not implemented I0803 22:18:22.878629 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0803 22:18:22.878717 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:22.878756 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:22.878841 20401 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0803 22:18:22.878905 20401 iamrole.go:89] ListRoles: { } I0803 22:18:22.878987 20401 internetgateways.go:101] DescribeInternetGateways: { } I0803 22:18:22.879054 20401 routetable.go:53] DescribeRouteTables: { } I0803 22:18:22.879089 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0803 22:18:22.879126 20401 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0803 22:18:22.879159 20401 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } autoscaling-config:lt-2 ok I0803 22:18:22.879747 20401 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0803 22:18:22.879808 20401 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } iam-instance-profile:masters.private-shared-ip.example.com ok I0803 22:18:22.879808 20401 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } autoscaling-group:nodes.private-shared-ip.example.com ok nat-gateway:nat-1 ok I0803 22:18:22.879888 20401 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } autoscaling-group:bastion.private-shared-ip.example.com ok I0803 22:18:22.879893 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0803 22:18:22.879940 20401 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } iam-instance-profile:bastions.private-shared-ip.example.com ok autoscaling-group:master-us-test-1a.masters.private-shared-ip.example.com ok I0803 22:18:22.879919 20401 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } load-balancer:bastion-private-shared-ip-eepmph ok I0803 22:18:22.880052 20401 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0803 22:18:22.880132 20401 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } iam-instance-profile:nodes.private-shared-ip.example.com ok I0803 22:18:22.879978 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } I0803 22:18:22.880124 20401 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } W0803 22:18:22.880196 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions load-balancer:api-private-shared-ip-exa-ohatqj ok I0803 22:18:22.879884 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:22.880225 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.private-shared-ip.example.com. ok security-group:sg-4 ok I0803 22:18:22.880276 20401 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0803 22:18:22.880375 20401 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0803 22:18:22.880437 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:22.880500 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0803 22:18:22.880631 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:22.880652 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0803 22:18:22.880694 20401 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0803 22:18:22.880748 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0803 22:18:22.880802 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W0803 22:18:22.880884 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:22.880898 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok autoscaling-config:lt-3 ok autoscaling-config:lt-1 ok I0803 22:18:22.881141 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0803 22:18:22.881187 20401 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-ip.example.com" } I0803 22:18:22.881300 20401 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0803 22:18:22.881318 20401 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0803 22:18:22.881356 20401 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-ip.example.com" } iam-role:masters.private-shared-ip.example.com ok I0803 22:18:22.881270 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } I0803 22:18:22.881397 20401 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-ip.example.com" } W0803 22:18:22.881442 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:22.881464 20401 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0803 22:18:22.881472 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } I0803 22:18:22.881482 20401 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } security-group:sg-5 ok I0803 22:18:22.881517 20401 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-ip.example.com" } iam-role:nodes.private-shared-ip.example.com ok I0803 22:18:22.881553 20401 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-ip.example.com" } I0803 22:18:22.881597 20401 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0803 22:18:22.881554 20401 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } I0803 22:18:22.881610 20401 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } subnet:subnet-1 ok I0803 22:18:22.881640 20401 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-ip.example.com" } iam-role:bastions.private-shared-ip.example.com ok I0803 22:18:22.881656 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:22.881726 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } W0803 22:18:22.881854 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:22.881873 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0803 22:18:22.881929 20401 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0803 22:18:22.882031 20401 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0803 22:18:22.882121 20401 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0803 22:18:22.882394 20401 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedIP (0.37s) === RUN TestLifecycleNodeTerminationHandlerQueueProcessor I0803 22:18:22.882652 20401 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:18:22.882915 20401 internetgateways.go:66] CreateInternetGateway: { } I0803 22:18:22.882966 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:18:22.883009 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:18:22.883059 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:18:22.883101 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:18:22.883131 20401 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:18:22.883170 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:18:22.883198 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:18:22.883254 20401 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster nthsqsresources.longclustername.example.com I0803 22:18:22.907634 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0803 22:18:22.910641 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:22.910736 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:22.952553 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:23.077654 20401 iamrole.go:89] ListRoles: { } I0803 22:18:23.079430 20401 executor.go:111] Tasks: 0 done / 99 total; 45 can run I0803 22:18:23.080022 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0803 22:18:23.080287 20401 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0803 22:18:23.080464 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0803 22:18:23.080520 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } I0803 22:18:23.080603 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:23.080637 20401 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I0803 22:18:23.080548 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0803 22:18:23.080733 20401 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } I0803 22:18:23.080797 20401 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] } I0803 22:18:23.080830 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] }] } I0803 22:18:23.081053 20401 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0803 22:18:23.081128 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0803 22:18:23.081361 20401 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0803 22:18:23.081510 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0803 22:18:23.081569 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:23.081630 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:23.081719 20401 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0803 22:18:23.081863 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0803 22:18:23.081936 20401 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0803 22:18:23.081975 20401 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0803 22:18:23.082006 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0803 22:18:23.082058 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:23.082118 20401 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0803 22:18:23.083722 20401 executor.go:111] Tasks: 45 done / 99 total; 24 can run I0803 22:18:23.083942 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0803 22:18:23.083910 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:23.084087 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:23.084243 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0803 22:18:23.084401 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0803 22:18:23.084461 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0803 22:18:23.084621 20401 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0803 22:18:23.084830 20401 iamrole.go:161] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0803 22:18:23.084805 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I0803 22:18:23.084957 20401 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0803 22:18:23.085377 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0803 22:18:23.085291 20401 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0803 22:18:23.085550 20401 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0803 22:18:23.085696 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] }] } I0803 22:18:23.086280 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I0803 22:18:23.088012 20401 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0803 22:18:23.088071 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0803 22:18:23.088178 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0803 22:18:23.088077 20401 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0803 22:18:23.088288 20401 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" } ] } I0803 22:18:23.088373 20401 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0803 22:18:23.088456 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0803 22:18:23.088521 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:23.088606 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0803 22:18:23.088698 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0803 22:18:23.088838 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:23.088917 20401 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0803 22:18:23.089074 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0803 22:18:23.089159 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:18:23.089839 20401 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\",\n \"sqs:DeleteMessage\",\n \"sqs:ReceiveMessage\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:CompleteLifecycleAction\",\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0803 22:18:23.091940 20401 executor.go:111] Tasks: 69 done / 99 total; 26 can run I0803 22:18:23.092190 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-4003to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0803 22:18:23.092398 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:23.092616 20401 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0803 22:18:23.092764 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-2382to4000-masters.nthsqsresources.longclustername.example.com" }] }] } I0803 22:18:23.093009 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0803 22:18:23.093160 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0803 22:18:23.093380 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-udp-1to65535-masters.nthsqsresources.longclustername.example.com" }] }] } I0803 22:18:23.093548 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:23.093639 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0803 22:18:23.093787 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:23.093859 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-1to2379-masters.nthsqsresources.longclustername.example.com" }] }] } I0803 22:18:23.094006 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0803 22:18:23.094144 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" }] }] } I0803 22:18:23.094277 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0803 22:18:23.094417 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0803 22:18:23.094549 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:23.094667 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:23.094801 20401 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0803 22:18:23.095110 20401 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0803 22:18:23.095208 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0803 22:18:23.095474 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:23.095636 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:23.095688 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0803 22:18:23.095728 20401 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0803 22:18:23.095882 20401 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0803 22:18:23.097245 20401 executor.go:111] Tasks: 95 done / 99 total; 2 can run I0803 22:18:23.097564 20401 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0803 22:18:23.097864 20401 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0803 22:18:23.097939 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:23.098000 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:23.098099 20401 dns.go:238] Pre-creating DNS records I0803 22:18:23.098201 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:23.104263 20401 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0803 22:18:23.107296 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:23.107395 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:23.146626 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:23.269795 20401 iamrole.go:89] ListRoles: { } I0803 22:18:23.271404 20401 executor.go:111] Tasks: 0 done / 99 total; 45 can run I0803 22:18:23.271819 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0803 22:18:23.272879 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0803 22:18:23.272981 20401 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0803 22:18:23.273314 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:23.273646 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0803 22:18:23.274282 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0803 22:18:23.274803 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:23.274923 20401 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0803 22:18:23.275087 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0803 22:18:23.275106 20401 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0803 22:18:23.279915 20401 executor.go:111] Tasks: 45 done / 99 total; 24 can run I0803 22:18:23.280244 20401 iamrole.go:161] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0803 22:18:23.280168 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0803 22:18:23.280413 20401 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:18:23.280494 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:23.280619 20401 iamrole.go:161] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0803 22:18:23.280664 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I0803 22:18:23.280887 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:23.280987 20401 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:18:23.281165 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0803 22:18:23.281363 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I0803 22:18:23.281589 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:23.281658 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0803 22:18:23.285309 20401 executor.go:111] Tasks: 69 done / 99 total; 26 can run I0803 22:18:23.285538 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:23.285676 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:23.285867 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:23.286186 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:23.286374 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:23.286625 20401 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0803 22:18:23.286826 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0803 22:18:23.289148 20401 executor.go:111] Tasks: 95 done / 99 total; 2 can run I0803 22:18:23.289429 20401 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0803 22:18:23.289594 20401 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0803 22:18:23.292588 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0803 22:18:23.292651 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0803 22:18:23.292689 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0803 22:18:23.292723 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0803 22:18:23.292766 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0803 22:18:23.292803 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0803 22:18:23.292836 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0803 22:18:23.292871 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0803 22:18:23.292907 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0803 22:18:23.292947 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0803 22:18:23.292985 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0803 22:18:23.293031 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0803 22:18:23.293063 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0803 22:18:23.293097 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0803 22:18:23.293132 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0803 22:18:23.293166 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0803 22:18:23.293203 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0803 22:18:23.293241 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0803 22:18:23.293280 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0803 22:18:23.293314 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0803 22:18:23.293346 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0803 22:18:23.293380 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0803 22:18:23.293418 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0803 22:18:23.293456 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0803 22:18:23.293494 20401 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0803 22:18:23.294979 20401 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0803 22:18:23.295007 20401 keypairs.go:95] DescribeKeyPairs: { } I0803 22:18:23.295047 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0803 22:18:23.295104 20401 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I0803 22:18:23.295163 20401 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0803 22:18:23.295221 20401 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0803 22:18:23.295268 20401 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0803 22:18:23.295325 20401 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0803 22:18:23.295369 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0803 22:18:23.295412 20401 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I0803 22:18:23.295500 20401 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0803 22:18:23.295560 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0803 22:18:23.295602 20401 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } W0803 22:18:23.295660 20401 api.go:53] PageSize not implemented I0803 22:18:23.295684 20401 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0803 22:18:23.295712 20401 loadbalancers.go:34] PageSize not implemented I0803 22:18:23.295732 20401 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0803 22:18:23.295757 20401 targetgroups.go:35] PageSize not implemented I0803 22:18:23.295776 20401 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0803 22:18:23.295833 20401 zones.go:93] ListHostedZonesPages { } I0803 22:18:23.295863 20401 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0803 22:18:23.295914 20401 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0803 22:18:23.295952 20401 iamrole.go:89] ListRoles: { } I0803 22:18:23.296002 20401 internetgateways.go:101] DescribeInternetGateways: { } I0803 22:18:23.296045 20401 routetable.go:53] DescribeRouteTables: { } I0803 22:18:23.296078 20401 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:nodes.nthsqsresources.longclustername.example.com ok I0803 22:18:23.296576 20401 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-group:master-us-test-1a.masters.nthsqsresources.longclustername.example.com ok I0803 22:18:23.296621 20401 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0803 22:18:23.296705 20401 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } iam-instance-profile:nodes.nthsqsresources.longclustername.example.com ok I0803 22:18:23.296651 20401 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0803 22:18:23.296740 20401 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } autoscaling-config:lt-2 ok I0803 22:18:23.296802 20401 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } iam-instance-profile:masters.nthsqsresources.longclustername.example.com ok I0803 22:18:23.296819 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0803 22:18:23.296898 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } I0803 22:18:23.296923 20401 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.nthsqsresources.longclustername.example.com. ok W0803 22:18:23.297134 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:23.297213 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0803 22:18:23.297275 20401 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0803 22:18:23.297350 20401 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0803 22:18:23.297492 20401 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0803 22:18:23.297524 20401 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-1 ok I0803 22:18:23.297605 20401 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0803 22:18:23.297692 20401 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0803 22:18:23.297768 20401 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0803 22:18:23.297893 20401 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0803 22:18:23.297922 20401 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.nthsqsresources.longclustername.example.com" } I0803 22:18:23.298016 20401 iamrole.go:180] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0803 22:18:23.298037 20401 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0803 22:18:23.298076 20401 iamrole.go:137] DeleteRole: { RoleName: "masters.nthsqsresources.longclustername.example.com" } iam-role:masters.nthsqsresources.longclustername.example.com ok I0803 22:18:23.298124 20401 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0803 22:18:23.298211 20401 iamrole.go:180] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0803 22:18:23.298228 20401 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0803 22:18:23.298266 20401 iamrole.go:137] DeleteRole: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } iam-role:nodes.nthsqsresources.longclustername.example.com ok I0803 22:18:23.298358 20401 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0803 22:18:23.298445 20401 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0803 22:18:23.298521 20401 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0803 22:18:23.298821 20401 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleNodeTerminationHandlerQueueProcessor (0.42s) === RUN TestGetFilters --- PASS: TestGetFilters (0.00s) === RUN TestGetInstanceSelectorOpts --- PASS: TestGetInstanceSelectorOpts (0.00s) === RUN TestValidateAllPrivateOrPublicSubnets --- PASS: TestValidateAllPrivateOrPublicSubnets (0.00s) === RUN TestValidateUserSubnetsWithClusterSubnets --- PASS: TestValidateUserSubnetsWithClusterSubnets (0.00s) === RUN TestCreateInstanceGroup --- PASS: TestCreateInstanceGroup (0.00s) === RUN TestDecorateWithInstanceGroupSpecs --- PASS: TestDecorateWithInstanceGroupSpecs (0.00s) === RUN TestDecorateWithMixedInstancesPolicy --- PASS: TestDecorateWithMixedInstancesPolicy (0.00s) === RUN TestDecorateWithClusterAutoscalerLabels --- PASS: TestDecorateWithClusterAutoscalerLabels (0.00s) === RUN TestNewTemplateContext --- PASS: TestNewTemplateContext (0.00s) PASS ok k8s.io/kops/cmd/kops 53.954s ? k8s.io/kops/cmd/kops/util [no test files] ? k8s.io/kops/cmd/kops-controller [no test files] ? k8s.io/kops/cmd/kops-controller/controllers [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/config [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/server [no test files] === RUN TestBuildProxyRequest === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/livez === RUN TestBuildProxyRequest/http://127.0.0.1:3990/healthz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ready === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz? === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude --- PASS: TestBuildProxyRequest (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/livez (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/healthz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ready (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz? (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude (0.00s) PASS ok k8s.io/kops/cmd/kube-apiserver-healthcheck 0.136s ? k8s.io/kops/cmd/nodeup [no test files] ? k8s.io/kops/dns-controller/cmd/dns-controller [no test files] === RUN TestAliasForNodesInRole --- PASS: TestAliasForNodesInRole (0.00s) === RUN TestEnsureDotSuffix --- PASS: TestEnsureDotSuffix (0.00s) === RUN TestParseZoneSpec --- PASS: TestParseZoneSpec (0.00s) === RUN TestParseZoneRules I0803 22:16:38.663050 3614 zonespec.go:75] No rules specified, will permit management of all zones --- PASS: TestParseZoneRules (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/dns 0.080s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/util 0.083s === RUN TestPodController I0803 22:16:58.550241 14113 pod.go:59] starting pod controller I0803 22:16:58.552781 14113 stoppable.go:62] shutting down controller --- PASS: TestPodController (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/watchers 0.033s === RUN TestEquivalent --- PASS: TestEquivalent (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider 0.060s Parsing flags. Getting new test interface. Running tests... === RUN TestZonesList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000117db0} route53_test.go:93: Got zone list: [0xc000117de0] route53_test.go:98: Got at least 1 zone in list:&{0xc00033ecc0 0xc000130080} --- PASS: TestZonesList (0.00s) === RUN TestZonesID route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000117eb0} route53_test.go:93: Got zone list: [0xc000117ee0] route53_test.go:98: Got at least 1 zone in list:&{0xc00033ecc0 0xc000130090} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess route53_test.go:79: Got zones {0xc000117f80} route53_test.go:170: Successfully added managed DNS zone: &{0xc00033eea0 0xc0001300a8} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003600e0} route53_test.go:93: Got zone list: [0xc000360110] route53_test.go:98: Got at least 1 zone in list:&{0xc00033ecc0 0xc0001300c0} route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000360210} route53_test.go:93: Got zone list: [0xc000360240] route53_test.go:98: Got at least 1 zone in list:&{0xc00033ecc0 0xc0001300e0} route53_test.go:187: Successfully added resource record set: {0xc00018e100 0xc000130100} --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003604c0} route53_test.go:93: Got zone list: [0xc0003604f0] route53_test.go:98: Got at least 1 zone in list:&{0xc00033ecc0 0xc000130188} route53_test.go:199: Successfully added resource record set: {0xc00018e280 0xc0001301a8} route53_test.go:118: Got 1 recordsets: [0xc000360690] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003607c0} route53_test.go:93: Got zone list: [0xc0003607f0] route53_test.go:98: Got at least 1 zone in list:&{0xc00033ecc0 0xc000130240} route53_test.go:221: Successfully added resource record set: {0xc00018e400 0xc000130260} route53_test.go:228: Correctly failed to add duplicate resource record {0xc00018e400 0xc000130260}: attempt to create duplicate rrset www11.example.com::A --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000360b50} route53_test.go:93: Got zone list: [0xc000360b80] route53_test.go:98: Got at least 1 zone in list:&{0xc00033ecc0 0xc000130318} route53_test.go:246: Successfully removed resource set {0xc00018e600 0xc000130338} after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000360e00} route53_test.go:93: Got zone list: [0xc000360e30] route53_test.go:98: Got at least 1 zone in list:&{0xc00033ecc0 0xc0001303c0} route53_test.go:264: Successfully removed resource set {0xc00018e780 0xc0001303e0} after adding route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003610e0} route53_test.go:93: Got zone list: [0xc000361110] route53_test.go:98: Got at least 1 zone in list:&{0xc00033ecc0 0xc000130470} commontests.go:192: Successfully added resource record set: {0xc00018e900 0xc000130490} commontests.go:45: Correctly replaced resource record {0xc00018e900 0xc000130490} -> {0xc00018ea00 0xc0001304e0} commontests.go:142: Got 1 recordsets: [0xc000361420] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003615f0} route53_test.go:93: Got zone list: [0xc000361620] route53_test.go:98: Got at least 1 zone in list:&{0xc00033ecc0 0xc000130590} commontests.go:192: Successfully added resource record set: {0xc00018ec80 0xc0001305b0} commontests.go:70: Correctly replaced resource record {0xc00018ec80 0xc0001305b0} -> {0xc00018ed80 0xc000130600} commontests.go:142: Got 1 recordsets: [0xc000361930] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000361b50} route53_test.go:93: Got zone list: [0xc000361b80] route53_test.go:98: Got at least 1 zone in list:&{0xc00033ecc0 0xc0001306c0} commontests.go:192: Successfully added resource record set: {0xc00018f000 0xc0001306e0} commontests.go:142: Got 2 recordsets: [0xc000361df0 0xc000361e10] commontests.go:142: Got 2 recordsets: [0xc000361e80 0xc000361ea0] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000370040} route53_test.go:93: Got zone list: [0xc000370070] route53_test.go:98: Got at least 1 zone in list:&{0xc00033ecc0 0xc0001307e0} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53 0.056s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53/stubs [no test files] === RUN TestZonesList --- PASS: TestZonesList (0.00s) === RUN TestAdd --- PASS: TestAdd (0.00s) === RUN TestRemove --- PASS: TestRemove (0.00s) === RUN TestNewZone --- PASS: TestNewZone (0.00s) === RUN TestNewResourceRecordSet --- PASS: TestNewResourceRecordSet (0.00s) === RUN TestResourceRecordChangeset --- PASS: TestResourceRecordChangeset (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/do 0.060s === RUN TestZonesList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000682398 0xc000685000} clouddns_test.go:71: Got zone list: [0xc0006be318] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00068ae40 0xc0006be2e8} --- PASS: TestZonesList (0.00s) === RUN TestZonesID clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000682398 0xc000685100} clouddns_test.go:71: Got zone list: [0xc0006be3c0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00068ae40 0xc0006be390} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess clouddns_test.go:131: Getting zones clouddns_test.go:58: Got zones {0xc000682398 0xc000685200} clouddns_test.go:133: Got zones, making new Zone clouddns_test.go:149: Successfully added managed DNS zone: &{0xc00068af80 0xc0006be468} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000682398 0xc0006852c0} clouddns_test.go:71: Got zone list: [0xc0006be510] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00068ae40 0xc0006be4e0} clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000682398 0xc0006853e0} clouddns_test.go:71: Got zone list: [0xc0006be630] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00068ae40 0xc0006be600} clouddns_test.go:165: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000682398 0xc000685520} clouddns_test.go:71: Got zone list: [0xc0006be780] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00068ae40 0xc0006be750} clouddns_test.go:176: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:96: Got 1 recordsets: [<(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000682398 0xc000685680} clouddns_test.go:71: Got zone list: [0xc0006be978] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00068ae40 0xc0006be948} clouddns_test.go:198: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:205: Correctly failed to add duplicate resource record <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>: attempt to insert duplicate rrset {www11.example.com [10.10.10.10 169.20.20.20] 180 A} --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000682398 0xc0006857e0} clouddns_test.go:71: Got zone list: [0xc0006beaf8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00068ae40 0xc0006beac8} clouddns_test.go:223: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000682398 0xc000685920} clouddns_test.go:71: Got zone list: [0xc0006bec48] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00068ae40 0xc0006bec18} clouddns_test.go:241: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000682398 0xc000685a80} clouddns_test.go:71: Got zone list: [0xc0006bedf8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00068ae40 0xc0006bedc8} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:45: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000682398 0xc000685be0} clouddns_test.go:71: Got zone list: [0xc0006bf0b0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00068ae40 0xc0006bf080} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:70: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000682398 0xc000685d60} clouddns_test.go:71: Got zone list: [0xc0006bf3c8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00068ae40 0xc0006bf398} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000682398 0xc000685f40} clouddns_test.go:71: Got zone list: [0xc0006bf788] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00068ae40 0xc0006bf758} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns 0.065s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/interfaces [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/stubs [no test files] === RUN TestContract --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/openstack/designate 0.072s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/rrstype [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/tests [no test files] ? k8s.io/kops/examples/kops-api-example [no test files] === RUN TestBootstarapBuilder_Simple --- PASS: TestBootstarapBuilder_Simple (0.00s) PASS ok k8s.io/kops/nodeup/pkg/bootstrap 0.080s === RUN TestBuildAzure --- PASS: TestBuildAzure (0.00s) === RUN TestBuildAWSCustomNodeIPFamilies --- PASS: TestBuildAWSCustomNodeIPFamilies (0.00s) === RUN TestContainerdBuilder_Docker_19_03_13 I0803 22:17:29.945111 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:29.945340 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:29.945383 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:29.945420 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:29.945474 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:29.945507 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:29.945523 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:29.945553 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:29.945581 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:29.952808 20432 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_13 (0.01s) === RUN TestContainerdBuilder_Docker_19_03_14 I0803 22:17:29.956235 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:29.956373 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:29.956399 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:29.956424 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:29.956450 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:29.956469 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:29.956486 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:29.956506 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:29.956519 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:29.957434 20432 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_14 (0.00s) === RUN TestContainerdBuilder_Simple I0803 22:17:29.960501 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:29.960619 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:29.960644 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:29.960665 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:29.960695 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:29.960714 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:29.960738 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:29.960760 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:29.960772 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:29.961662 20432 zones.go:93] ListHostedZonesPages { } I0803 22:17:29.963022 20432 assetstore.go:108] Matching assets for "^bin/(containerd|ctr)": I0803 22:17:29.963040 20432 assetstore.go:111] bin/containerd I0803 22:17:29.963048 20432 assetstore.go:111] bin/containerd-shim I0803 22:17:29.963056 20432 assetstore.go:111] bin/containerd-shim-runc-v1 I0803 22:17:29.963063 20432 assetstore.go:111] bin/containerd-shim-runc-v2 I0803 22:17:29.963086 20432 assetstore.go:111] bin/containerd-stress I0803 22:17:29.963096 20432 assetstore.go:111] bin/ctr I0803 22:17:29.963132 20432 assetstore.go:108] Matching assets for "/runc\\.(amd64|arm64)$": I0803 22:17:29.963145 20432 assetstore.go:111] https://github.com/opencontainers/runc/releases/download/v1.1.0/runc.amd64 --- PASS: TestContainerdBuilder_Simple (0.01s) === RUN TestContainerdBuilder_Flatcar I0803 22:17:29.965975 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:29.966097 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:29.966121 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:29.966144 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:29.966167 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:29.966188 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:29.966203 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:29.966222 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:29.966235 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:29.967088 20432 zones.go:93] ListHostedZonesPages { } I0803 22:17:29.968326 20432 containerd.go:58] Detected Flatcar; won't install containerd --- PASS: TestContainerdBuilder_Flatcar (0.00s) === RUN TestContainerdBuilder_SkipInstall I0803 22:17:29.969156 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:29.969282 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:29.969305 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:29.969325 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:29.969348 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:29.969410 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:29.969425 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:29.969442 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:29.969455 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:29.970345 20432 zones.go:93] ListHostedZonesPages { } I0803 22:17:29.971672 20432 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestContainerdBuilder_SkipInstall (0.00s) === RUN TestContainerdBuilder_BuildFlags --- PASS: TestContainerdBuilder_BuildFlags (0.00s) === RUN TestContainerdConfig --- PASS: TestContainerdConfig (0.00s) === RUN TestAppendGPURuntimeContainerdConfig --- PASS: TestAppendGPURuntimeContainerdConfig (0.00s) === RUN TestDockerBuilder_19_03_11 I0803 22:17:29.972232 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:29.972348 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:29.972373 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:29.972393 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:29.972415 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:29.972453 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:29.972483 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:29.972504 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:29.972518 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:29.973851 20432 zones.go:93] ListHostedZonesPages { } I0803 22:17:29.977107 20432 assetstore.go:108] Matching assets for "^docker/": I0803 22:17:29.977133 20432 assetstore.go:111] docker/containerd I0803 22:17:29.977142 20432 assetstore.go:111] docker/containerd-shim I0803 22:17:29.977149 20432 assetstore.go:111] docker/ctr I0803 22:17:29.977156 20432 assetstore.go:111] docker/docker I0803 22:17:29.977163 20432 assetstore.go:111] docker/docker-init I0803 22:17:29.977170 20432 assetstore.go:111] docker/docker-proxy I0803 22:17:29.977177 20432 assetstore.go:111] docker/dockerd I0803 22:17:29.977184 20432 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_19_03_11 (0.01s) === RUN TestDockerBuilder_LogFlags I0803 22:17:29.980079 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:29.980236 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:29.980266 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:29.980294 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:29.980323 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:29.980370 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:29.980386 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:29.980408 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:29.980423 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:29.981468 20432 zones.go:93] ListHostedZonesPages { } I0803 22:17:29.982907 20432 assetstore.go:108] Matching assets for "^docker/": I0803 22:17:29.982923 20432 assetstore.go:111] docker/containerd I0803 22:17:29.982931 20432 assetstore.go:111] docker/containerd-shim I0803 22:17:29.982938 20432 assetstore.go:111] docker/ctr I0803 22:17:29.982945 20432 assetstore.go:111] docker/docker I0803 22:17:29.982953 20432 assetstore.go:111] docker/docker-init I0803 22:17:29.982961 20432 assetstore.go:111] docker/docker-proxy I0803 22:17:29.982968 20432 assetstore.go:111] docker/dockerd I0803 22:17:29.982975 20432 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_LogFlags (0.01s) === RUN TestDockerBuilder_SkipInstall I0803 22:17:29.985679 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:29.985802 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:29.985826 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:29.985847 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:29.985873 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:29.985913 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:29.985930 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:29.985949 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:29.985961 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:29.986823 20432 zones.go:93] ListHostedZonesPages { } I0803 22:17:29.988178 20432 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestDockerBuilder_SkipInstall (0.00s) === RUN TestDockerBuilder_HealthCheck I0803 22:17:29.988321 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:29.988443 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:29.988467 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:29.988502 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:29.988528 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:29.988567 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:29.988585 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:29.988604 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:29.988618 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:29.989518 20432 zones.go:93] ListHostedZonesPages { } I0803 22:17:29.990859 20432 assetstore.go:108] Matching assets for "^docker/": I0803 22:17:29.990875 20432 assetstore.go:111] docker/docker I0803 22:17:29.990883 20432 assetstore.go:111] docker/docker-containerd I0803 22:17:29.990890 20432 assetstore.go:111] docker/docker-containerd-ctr I0803 22:17:29.990897 20432 assetstore.go:111] docker/docker-containerd-shim I0803 22:17:29.990904 20432 assetstore.go:111] docker/docker-init I0803 22:17:29.990911 20432 assetstore.go:111] docker/docker-proxy I0803 22:17:29.990918 20432 assetstore.go:111] docker/docker-runc I0803 22:17:29.990925 20432 assetstore.go:111] docker/dockerd --- PASS: TestDockerBuilder_HealthCheck (0.01s) === RUN TestDockerBuilder_BuildFlags --- PASS: TestDockerBuilder_BuildFlags (0.00s) === RUN TestContainerdHooksBuilder I0803 22:17:29.994767 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:29.994891 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:29.994914 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:29.994936 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:29.994959 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:29.994979 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:29.994995 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:29.995015 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:29.995027 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:29.997066 20432 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdHooksBuilder (0.00s) === RUN TestDockerHooksBuilder I0803 22:17:29.999430 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:29.999572 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:29.999598 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:29.999620 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:29.999645 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:29.999666 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:29.999682 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:29.999705 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:29.999721 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:30.001339 20432 zones.go:93] ListHostedZonesPages { } --- PASS: TestDockerHooksBuilder (0.00s) === RUN TestKopsControllerBuilder I0803 22:17:30.004130 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:30.004253 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:30.004279 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:30.004304 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:30.004330 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:30.004352 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:30.004376 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:30.004396 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:30.004410 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:30.005924 20432 zones.go:93] ListHostedZonesPages { } --- PASS: TestKopsControllerBuilder (0.01s) === RUN Test_KubeAPIServer_BuildFlags --- PASS: Test_KubeAPIServer_BuildFlags (0.00s) === RUN TestKubeAPIServerBuilder I0803 22:17:30.012435 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:30.012569 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:30.012611 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:30.012638 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:30.012662 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:30.012683 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:30.012698 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:30.012719 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:30.012741 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:30.017839 20432 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilder (0.02s) === RUN TestDeddicatedAPIServerBuilder I0803 22:17:30.033005 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:30.033246 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:30.033297 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:30.033345 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:30.033396 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:30.033438 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:30.033466 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:30.033504 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:30.033534 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:30.036223 20432 zones.go:93] ListHostedZonesPages { } --- PASS: TestDeddicatedAPIServerBuilder (0.01s) === RUN TestWithoutEtcdEventsAPIServerBuilder I0803 22:17:30.046215 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:30.046401 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:30.046441 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:30.046475 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:30.046521 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:30.046553 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:30.046577 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:30.046607 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:30.046628 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:30.048714 20432 zones.go:93] ListHostedZonesPages { } --- PASS: TestWithoutEtcdEventsAPIServerBuilder (0.01s) === RUN TestAwsIamAuthenticator I0803 22:17:30.058156 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:30.058340 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:30.058370 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:30.058410 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:30.058440 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:30.058487 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:30.058511 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:30.058533 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:30.058547 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:30.060598 20432 zones.go:93] ListHostedZonesPages { } --- PASS: TestAwsIamAuthenticator (0.01s) === RUN TestKubeAPIServerBuilderAMD64 I0803 22:17:30.069336 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:30.069471 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:30.069498 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:30.069524 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:30.069552 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:30.069575 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:30.069594 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:30.069617 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:30.069633 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:30.071303 20432 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderAMD64 (1.53s) === RUN TestKubeAPIServerBuilderARM64 I0803 22:17:31.594869 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:31.595042 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:31.595079 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:31.595111 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:31.595145 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:31.595172 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:31.595194 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:31.595220 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:31.595237 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:31.596914 20432 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderARM64 (0.08s) === RUN TestKubeControllerManagerBuilder I0803 22:17:31.671441 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:31.671719 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:31.671772 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:31.671823 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:31.671874 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:31.671912 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:31.671941 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:31.671978 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:31.672005 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:31.674823 20432 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilder (0.01s) === RUN TestKubeControllerManagerBuilderAMD64 I0803 22:17:31.685267 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:31.685514 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:31.685560 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:31.685599 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:31.685642 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:31.685676 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:31.685706 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:31.685742 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:31.685766 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:31.688390 20432 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderAMD64 (0.08s) === RUN TestKubeControllerManagerBuilderARM64 I0803 22:17:31.761147 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:31.761372 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:31.761415 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:31.761458 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:31.761505 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:31.761576 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:31.761613 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:31.761650 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:31.761675 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:31.764299 20432 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderARM64 (0.07s) === RUN TestKubeProxyBuilder_buildPod === RUN TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 --- PASS: TestKubeProxyBuilder_buildPod (0.00s) --- PASS: TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 (0.00s) === RUN TestKubeProxyBuilder I0803 22:17:31.836098 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:31.836261 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:31.836294 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:31.836323 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:31.836355 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:31.836382 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:31.836398 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:31.836421 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:31.836437 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:31.838026 20432 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilder (0.01s) === RUN TestKubeProxyBuilderAMD64 I0803 22:17:31.841618 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:31.841744 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:31.841769 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:31.841791 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:31.841815 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:31.841836 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:31.841854 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:31.841874 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:31.841887 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:31.843424 20432 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderAMD64 (0.07s) === RUN TestKubeProxyBuilderARM64 I0803 22:17:31.914129 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:31.914324 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:31.914358 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:31.914392 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:31.914428 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:31.914453 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:31.914476 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:31.914502 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:31.914517 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:31.916190 20432 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderARM64 (0.07s) === RUN TestParseDefault --- PASS: TestParseDefault (0.00s) === RUN TestParse --- PASS: TestParse (0.00s) === RUN TestKubeSchedulerBuilder I0803 22:17:31.986143 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:31.986395 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:31.986445 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:31.986494 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:31.986555 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:31.986608 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:31.986640 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:31.986690 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:31.986719 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:31.989626 20432 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeSchedulerBuilder (0.01s) === RUN TestKubeSchedulerBuilderAMD64 I0803 22:17:31.998859 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:31.999138 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:31.999193 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:31.999243 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:31.999294 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:31.999383 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:31.999416 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:31.999453 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:31.999487 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:32.002790 20432 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeSchedulerBuilderAMD64 (0.07s) === RUN TestKubeSchedulerBuilderARM64 I0803 22:17:32.071157 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:32.071356 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:32.071393 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:32.071424 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:32.071457 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:32.071485 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:32.071507 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:32.071533 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:32.071551 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:32.073369 20432 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeSchedulerBuilderARM64 (0.08s) === RUN TestKubectlBuilder I0803 22:17:32.150869 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:32.151266 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:32.151349 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:32.151425 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:32.151510 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:32.151584 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:32.151643 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:32.151714 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:32.151764 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:32.156299 20432 zones.go:93] ListHostedZonesPages { } I0803 22:17:32.160797 20432 assetstore.go:159] Resolved asset kubectl: to /path/to/kubectl/asset --- PASS: TestKubectlBuilder (0.01s) === RUN Test_InstanceGroupKubeletMerge --- PASS: Test_InstanceGroupKubeletMerge (0.00s) === RUN TestTaintsApplied --- PASS: TestTaintsApplied (0.00s) === RUN Test_RunKubeletBuilder I0803 22:17:32.164170 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:32.164408 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:32.164493 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:32.164549 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:32.164617 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:32.164667 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:32.164714 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:32.164761 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:32.164795 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:32.167857 20432 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilder (0.01s) === RUN Test_RunKubeletBuilderWarmPool I0803 22:17:32.175133 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:32.175337 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:32.175405 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:32.175494 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:32.175547 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:32.175586 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:32.175620 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:32.175660 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:32.175688 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:32.178055 20432 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilderWarmPool (0.01s) === RUN Test_BuildComponentConfigFile --- PASS: Test_BuildComponentConfigFile (0.00s) === RUN TestProtokubeBuilder I0803 22:17:32.184341 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:32.184522 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:32.184556 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:32.184594 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:32.184628 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:32.184653 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:32.184670 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:32.184694 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:32.184712 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:32.186516 20432 zones.go:93] ListHostedZonesPages { } I0803 22:17:32.188121 20432 assetstore.go:108] Matching assets for "protokube$": I0803 22:17:32.188139 20432 assetstore.go:111] /opt/kops/bin/protokube I0803 22:17:32.188150 20432 assetstore.go:129] Found single matching asset for expr "protokube$": "protokube" I0803 22:17:32.188169 20432 assetstore.go:108] Matching assets for "channels$": I0803 22:17:32.188178 20432 assetstore.go:111] /opt/kops/bin/channels I0803 22:17:32.188187 20432 assetstore.go:129] Found single matching asset for expr "channels$": "channels" --- PASS: TestProtokubeBuilder (0.01s) === RUN TestSecretBuilder I0803 22:17:32.189547 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:32.189676 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:32.189701 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:32.189724 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:32.189749 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:32.189771 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:32.189787 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:32.189806 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:32.189819 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:32.191329 20432 zones.go:93] ListHostedZonesPages { } --- PASS: TestSecretBuilder (0.00s) === RUN TestUpdateServiceBuilderAutomaticUpgrade I0803 22:17:32.193447 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:32.193576 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:32.193602 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:32.193624 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:32.193649 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:32.193670 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:32.193688 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:32.193709 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:32.193722 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:32.195177 20432 zones.go:93] ListHostedZonesPages { } I0803 22:17:32.196715 20432 update_service.go:96] Detected OS {deb ubuntu focal 20.04}; installing unattended-upgrades package --- PASS: TestUpdateServiceBuilderAutomaticUpgrade (0.00s) === RUN TestUpdateServiceBuilderExternal I0803 22:17:32.197114 20432 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:32.197240 20432 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:32.197265 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:32.197286 20432 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:32.197311 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:32.197332 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:32.197350 20432 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:32.197371 20432 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:32.197385 20432 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:32.198844 20432 zones.go:93] ListHostedZonesPages { } I0803 22:17:32.200369 20432 update_service.go:91] UpdatePolicy requests automatic updates; skipping installation of package "unattended-upgrades" --- PASS: TestUpdateServiceBuilderExternal (0.00s) PASS ok k8s.io/kops/nodeup/pkg/model 2.322s ? k8s.io/kops/nodeup/pkg/model/dns [no test files] ? k8s.io/kops/nodeup/pkg/model/networking [no test files] ? k8s.io/kops/nodeup/pkg/model/resources [no test files] ? k8s.io/kops/pkg/acls [no test files] ? k8s.io/kops/pkg/acls/gce [no test files] === RUN Test_Strategy --- PASS: Test_Strategy (0.00s) === RUN Test_In_StateStore --- PASS: Test_In_StateStore (0.00s) PASS ok k8s.io/kops/pkg/acls/s3 0.095s === RUN TestWarmPoolSpec_IsEnabled === RUN TestWarmPoolSpec_IsEnabled/nil === RUN TestWarmPoolSpec_IsEnabled/empty === RUN TestWarmPoolSpec_IsEnabled/1 === RUN TestWarmPoolSpec_IsEnabled/0 --- PASS: TestWarmPoolSpec_IsEnabled (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/nil (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/empty (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/1 (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/0 (0.00s) === RUN TestWarmPoolSpec_ResolveDefaults === RUN TestWarmPoolSpec_ResolveDefaults/MinSize === RUN TestWarmPoolSpec_ResolveDefaults/MaxSize === RUN TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook --- PASS: TestWarmPoolSpec_ResolveDefaults (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MinSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MaxSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook (0.00s) === RUN Test_ParseInstanceGroupRole --- PASS: Test_ParseInstanceGroupRole (0.00s) === RUN TestParseConfigYAML === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{}" --- PASS: TestParseConfigYAML (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{}" (0.00s) === RUN TestWeaveParseConfigYAML === RUN TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" === RUN TestWeaveParseConfigYAML/"networking:_{}" --- PASS: TestWeaveParseConfigYAML (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{}" (0.00s) === RUN Test_SemverOrdering --- PASS: Test_SemverOrdering (0.00s) PASS ok k8s.io/kops/pkg/apis/kops 0.026s === RUN TestRoundTripTypes --- PASS: TestRoundTripTypes (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/install 0.057s === RUN TestUseCiliumEtcd --- PASS: TestUseCiliumEtcd (0.00s) === RUN Test_FindSubnet --- PASS: Test_FindSubnet (0.00s) === RUN Test_FindZonesForInstanceGroup --- PASS: Test_FindZonesForInstanceGroup (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/model 0.080s ? k8s.io/kops/pkg/apis/kops/registry [no test files] === RUN TestGetNodeRole === RUN TestGetNodeRole/RoleNone === RUN TestGetNodeRole/RoleNewerLabel === RUN TestGetNodeRole/RoleOlderLabel --- PASS: TestGetNodeRole (0.00s) --- PASS: TestGetNodeRole/RoleNone (0.00s) --- PASS: TestGetNodeRole/RoleNewerLabel (0.00s) --- PASS: TestGetNodeRole/RoleOlderLabel (0.00s) === RUN Test_ParseKubernetesVersion === RUN Test_ParseKubernetesVersion/1.3.7 === RUN Test_ParseKubernetesVersion/v1.4.0-beta.8 === RUN Test_ParseKubernetesVersion/1.5.0 === RUN Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ === RUN Test_ParseKubernetesVersion/#00 E0803 22:16:43.872544 6612 versions.go:96] unable to parse Kubernetes version "" === RUN Test_ParseKubernetesVersion/abc E0803 22:16:43.872641 6612 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: Test_ParseKubernetesVersion (0.00s) --- PASS: Test_ParseKubernetesVersion/1.3.7 (0.00s) --- PASS: Test_ParseKubernetesVersion/v1.4.0-beta.8 (0.00s) --- PASS: Test_ParseKubernetesVersion/1.5.0 (0.00s) --- PASS: Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ (0.00s) --- PASS: Test_ParseKubernetesVersion/#00 (0.00s) --- PASS: Test_ParseKubernetesVersion/abc (0.00s) === RUN Test_IsKubernetesGTEWithPatch --- PASS: Test_IsKubernetesGTEWithPatch (0.00s) === RUN Test_IsKubernetesGTEWithoutPatch --- PASS: Test_IsKubernetesGTEWithoutPatch (0.00s) === RUN Test_IsKubernetesGTEWithPre --- PASS: Test_IsKubernetesGTEWithPre (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/util 0.021s ? k8s.io/kops/pkg/apis/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha3 [no test files] === RUN TestAWSValidateExternalCloudConfig --- PASS: TestAWSValidateExternalCloudConfig (0.00s) === RUN TestValidateInstanceGroupSpec I0803 22:17:18.658843 19572 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0803 22:17:18.658934 19572 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0803 22:17:18.658965 19572 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0803 22:17:18.658990 19572 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0803 22:17:18.659013 19572 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestValidateInstanceGroupSpec (0.00s) === RUN TestMixedInstancePolicies I0803 22:17:18.659173 19572 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0803 22:17:18.659204 19572 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0803 22:17:18.659226 19572 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0803 22:17:18.659248 19572 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0803 22:17:18.659275 19572 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0803 22:17:18.659299 19572 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0803 22:17:18.659327 19572 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0803 22:17:18.659349 19572 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0803 22:17:18.659373 19572 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0803 22:17:18.659397 19572 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0803 22:17:18.659418 19572 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0803 22:17:18.659443 19572 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0803 22:17:18.659466 19572 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0803 22:17:18.659486 19572 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0803 22:17:18.659506 19572 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0803 22:17:18.659537 19572 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0803 22:17:18.659563 19572 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0803 22:17:18.659584 19572 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0803 22:17:18.659604 19572 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestMixedInstancePolicies (0.00s) === RUN TestInstanceMetadataOptions I0803 22:17:18.659738 19572 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } I0803 22:17:18.659788 19572 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } --- PASS: TestInstanceMetadataOptions (0.00s) === RUN TestLoadBalancerSubnets --- PASS: TestLoadBalancerSubnets (0.00s) === RUN TestAWSAuthentication --- PASS: TestAWSAuthentication (0.00s) === RUN TestAWSAdditionalRoutes --- PASS: TestAWSAdditionalRoutes (0.00s) === RUN TestValidEtcdChanges --- PASS: TestValidEtcdChanges (0.00s) === RUN TestValidateInstanceProfile --- PASS: TestValidateInstanceProfile (0.00s) === RUN TestValidMasterInstanceGroup --- PASS: TestValidMasterInstanceGroup (0.00s) === RUN TestValidBootDevice --- PASS: TestValidBootDevice (0.00s) === RUN TestValidNodeLabels --- PASS: TestValidNodeLabels (0.00s) === RUN TestValidateIGCloudLabels --- PASS: TestValidateIGCloudLabels (0.00s) === RUN TestIGCloudLabelIsIGName --- PASS: TestIGCloudLabelIsIGName (0.00s) === RUN TestValidTaints --- PASS: TestValidTaints (0.00s) === RUN TestIGUpdatePolicy === RUN TestIGUpdatePolicy/missing === RUN TestIGUpdatePolicy/automatic === RUN TestIGUpdatePolicy/external === RUN TestIGUpdatePolicy/empty === RUN TestIGUpdatePolicy/unknown --- PASS: TestIGUpdatePolicy (0.00s) --- PASS: TestIGUpdatePolicy/missing (0.00s) --- PASS: TestIGUpdatePolicy/automatic (0.00s) --- PASS: TestIGUpdatePolicy/external (0.00s) --- PASS: TestIGUpdatePolicy/empty (0.00s) --- PASS: TestIGUpdatePolicy/unknown (0.00s) === RUN TestValidInstanceGroup --- PASS: TestValidInstanceGroup (0.00s) === RUN Test_ValidateTopology --- PASS: Test_ValidateTopology (0.00s) === RUN Test_Validate_DNS --- PASS: Test_Validate_DNS (0.00s) === RUN TestValidateCIDR --- PASS: TestValidateCIDR (0.00s) === RUN TestValidateSubnets --- PASS: TestValidateSubnets (0.00s) === RUN TestValidateKubeAPIServer --- PASS: TestValidateKubeAPIServer (0.00s) === RUN Test_Validate_DockerConfig_Storage --- PASS: Test_Validate_DockerConfig_Storage (0.00s) === RUN Test_Validate_Networking_Flannel --- PASS: Test_Validate_Networking_Flannel (0.00s) === RUN Test_Validate_AdditionalPolicies --- PASS: Test_Validate_AdditionalPolicies (0.00s) === RUN Test_Validate_Calico === RUN Test_Validate_Calico/empty_specs === RUN Test_Validate_Calico/positive_Typha_replica_count === RUN Test_Validate_Calico/negative_Typha_replica_count === RUN Test_Validate_Calico/with_etcd_version === RUN Test_Validate_Calico/IPv4_autodetection_method === RUN Test_Validate_Calico/IPv6_autodetection_method === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name === RUN Test_Validate_Calico/AWS_source/destination_checks_off === RUN Test_Validate_Calico/AWS_source/destination_checks_enabled === RUN Test_Validate_Calico/AWS_source/destination_checks_disabled === RUN Test_Validate_Calico/AWS_source/destination_checks_left_as_is === RUN Test_Validate_Calico/encapsulation_none_with_IPv4 === RUN Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 === RUN Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 === RUN Test_Validate_Calico/unknown_Calico_IPIP_mode === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPv6_without_encapsulation --- PASS: Test_Validate_Calico (0.00s) --- PASS: Test_Validate_Calico/empty_specs (0.00s) --- PASS: Test_Validate_Calico/positive_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/negative_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/with_etcd_version (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_off (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_enabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_disabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_left_as_is (0.00s) --- PASS: Test_Validate_Calico/encapsulation_none_with_IPv4 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/unknown_Calico_IPIP_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPv6_without_encapsulation (0.00s) === RUN Test_Validate_Cilium --- PASS: Test_Validate_Cilium (0.00s) === RUN Test_Validate_RollingUpdate --- PASS: Test_Validate_RollingUpdate (0.00s) === RUN Test_Validate_NodeLocalDNS --- PASS: Test_Validate_NodeLocalDNS (0.00s) === RUN Test_Validate_CloudConfiguration === RUN Test_Validate_CloudConfiguration/neither === RUN Test_Validate_CloudConfiguration/all_false === RUN Test_Validate_CloudConfiguration/all_true === RUN Test_Validate_CloudConfiguration/os_false === RUN Test_Validate_CloudConfiguration/os_true === RUN Test_Validate_CloudConfiguration/all_false,_os_false === RUN Test_Validate_CloudConfiguration/all_false,_os_true === RUN Test_Validate_CloudConfiguration/all_true,_os_false === RUN Test_Validate_CloudConfiguration/all_true,_os_true --- PASS: Test_Validate_CloudConfiguration (0.00s) --- PASS: Test_Validate_CloudConfiguration/neither (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_true (0.00s) === RUN TestValidateSAExternalPermissions === RUN TestValidateSAExternalPermissions/Duplicate_SA === RUN TestValidateSAExternalPermissions/Missing_permissions === RUN TestValidateSAExternalPermissions/Setting_both_arn_and_inline === RUN TestValidateSAExternalPermissions/Empty_SA_name === RUN TestValidateSAExternalPermissions/Empty_SA_namespace --- PASS: TestValidateSAExternalPermissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Duplicate_SA (0.00s) --- PASS: TestValidateSAExternalPermissions/Missing_permissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Setting_both_arn_and_inline (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_name (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_namespace (0.00s) === RUN Test_Validate_Nvidia_Cluster --- PASS: Test_Validate_Nvidia_Cluster (0.00s) === RUN Test_Validate_Nvidia_Ig --- PASS: Test_Validate_Nvidia_Ig (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/validation 0.065s ? k8s.io/kops/pkg/apis/nodeup [no test files] === RUN TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub W0803 22:16:58.302380 13618 builder.go:215] failed to digest image "proxy.example.com/weaveworks/weave-kube" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub (0.81s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub W0803 22:16:58.303432 13618 builder.go:215] failed to digest image "proxy.example.com/debian" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL W0803 22:16:58.304521 13618 builder.go:215] failed to digest image "proxy.example.com/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL W0803 22:16:58.305538 13618 builder.go:215] failed to digest image "proxy.example.com/google_containers/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags W0803 22:16:58.306640 13618 builder.go:215] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags (0.00s) === RUN TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges W0803 22:16:58.307679 13618 builder.go:215] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" W0803 22:16:58.308694 13618 builder.go:215] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges (0.00s) === RUN TestRemapEmptySection --- PASS: TestRemapEmptySection (0.00s) === RUN Test_BuildVFSPath E0803 22:16:58.309376 13618 copyfile.go:205] Unable to determine VFS path from supplied URL: https://foo/k8s-for-greeks-kops/kubernetes-release/release/v1.7.2/bin/linux/amd64/kubectl E0803 22:16:58.309390 13618 copyfile.go:206] S3, Google Cloud Storage, and File Paths are supported. E0803 22:16:58.309398 13618 copyfile.go:207] For S3, please make sure that the supplied file repository URL adhere to S3 naming conventions, https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region. E0803 22:16:58.309410 13618 copyfile.go:208] For GCS, please make sure that the supplied file repository URL adheres to https://storage.googleapis.com/ --- PASS: Test_BuildVFSPath (0.00s) PASS ok k8s.io/kops/pkg/assets 0.852s === RUN TestBackoff --- PASS: TestBackoff (0.00s) PASS ok k8s.io/kops/pkg/backoff 0.039s ? k8s.io/kops/pkg/bootstrap [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/simple [no test files] ? k8s.io/kops/pkg/client/simple/api [no test files] === RUN TestSSHCredentialStoreOnConfigBase --- PASS: TestSSHCredentialStoreOnConfigBase (0.00s) === RUN TestSSHCredentialStoreOnOwnCFS --- PASS: TestSSHCredentialStoreOnOwnCFS (0.00s) PASS ok k8s.io/kops/pkg/client/simple/vfsclientset 0.057s === RUN TestToAzureVMName === RUN TestToAzureVMName/test_case_0 === RUN TestToAzureVMName/test_case_1 --- PASS: TestToAzureVMName (0.00s) --- PASS: TestToAzureVMName/test_case_0 (0.00s) --- PASS: TestToAzureVMName/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/cloudinstances 0.143s ? k8s.io/kops/pkg/clouds [no test files] ? k8s.io/kops/pkg/clusteraddons [no test files] === RUN TestSetClusterBadInput --- PASS: TestSetClusterBadInput (0.00s) === RUN TestSetClusterFields --- PASS: TestSetClusterFields (0.01s) === RUN TestSetCiliumFields --- PASS: TestSetCiliumFields (0.00s) === RUN TestSetInstanceGroupsBadInput --- PASS: TestSetInstanceGroupsBadInput (0.00s) === RUN TestSetInstanceGroupsFields --- PASS: TestSetInstanceGroupsFields (0.00s) === RUN TestUnsetClusterBadInput --- PASS: TestUnsetClusterBadInput (0.00s) === RUN TestUnsetClusterFields --- PASS: TestUnsetClusterFields (0.00s) === RUN TestUnsetCiliumFields --- PASS: TestUnsetCiliumFields (0.00s) === RUN TestUnsetInstanceGroupsBadInput --- PASS: TestUnsetInstanceGroupsBadInput (0.00s) === RUN TestUnsetInstanceGroupsFields --- PASS: TestUnsetInstanceGroupsFields (0.00s) PASS ok k8s.io/kops/pkg/commands 0.079s ? k8s.io/kops/pkg/commands/commandutils [no test files] ? k8s.io/kops/pkg/commands/helpers [no test files] === RUN TestGetStructVal --- PASS: TestGetStructVal (0.00s) === RUN TestWrongStructField --- PASS: TestWrongStructField (0.00s) PASS ok k8s.io/kops/pkg/configbuilder 0.095s ? k8s.io/kops/pkg/configserver [no test files] === RUN Test_Diff_1 --- PASS: Test_Diff_1 (0.00s) === RUN Test_Diff_2 --- PASS: Test_Diff_2 (0.00s) === RUN Test_Diff_3 --- PASS: Test_Diff_3 (0.00s) === RUN Test_Diff_ChangedLine --- PASS: Test_Diff_ChangedLine (0.00s) === RUN Test_Diff_4 --- PASS: Test_Diff_4 (0.00s) PASS ok k8s.io/kops/pkg/diff 0.065s === RUN TestIsGossipHostname --- PASS: TestIsGossipHostname (0.00s) PASS ok k8s.io/kops/pkg/dns 0.050s ? k8s.io/kops/pkg/dump [no test files] === RUN TestHasExtraFields --- PASS: TestHasExtraFields (0.02s) PASS ok k8s.io/kops/pkg/edit 0.074s === RUN TestFlagToFalse I0803 22:16:38.922473 3882 featureflag_test.go:33] Created flag Unittest1 I0803 22:16:38.922531 3882 featureflag.go:162] FeatureFlag "UnitTest1"=false I0803 22:16:38.922547 3882 featureflag.go:162] FeatureFlag "UnitTest1"=true --- PASS: TestFlagToFalse (0.00s) === RUN TestSetenv I0803 22:16:38.922612 3882 featureflag.go:162] FeatureFlag "UnitTest2"=false --- PASS: TestSetenv (0.00s) PASS ok k8s.io/kops/pkg/featureflag 0.088s === RUN TestBuildKCMFlags --- PASS: TestBuildKCMFlags (0.00s) === RUN TestKubeletConfigSpec --- PASS: TestKubeletConfigSpec (0.00s) === RUN TestBuildAPIServerFlags --- PASS: TestBuildAPIServerFlags (0.00s) PASS ok k8s.io/kops/pkg/flagbuilder 0.034s === RUN TestRenderInstanceGroupZones W0803 22:16:57.362654 13450 instancegroup.go:42] error fetching zones for instancegroup: cannot find subnet "badsubnet" (declared in instance group "", not found in cluster) --- PASS: TestRenderInstanceGroupZones (0.00s) === RUN TestRenderInstanceGroupSubnets --- PASS: TestRenderInstanceGroupSubnets (0.00s) PASS ok k8s.io/kops/pkg/formatter 0.114s === RUN TestRollingUpdateDisabledSurgeOS rollingupdate_os_test.go:100: Failing in new release PR when build is not yet published --- SKIP: TestRollingUpdateDisabledSurgeOS (0.00s) === RUN TestRollingUpdateAllNeedUpdate I0803 22:17:38.882258 20566 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0803 22:17:38.882311 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:38.883468 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:38.883495 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:38.888703 20566 instancegroups.go:503] Cluster validated. I0803 22:17:38.888722 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:38.888731 20566 instancegroups.go:503] Cluster validated. I0803 22:17:38.888738 20566 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0803 22:17:38.892641 20566 instancegroups.go:400] Draining the node: "master-1a.local". I0803 22:17:38.893477 20566 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0803 22:17:38.893517 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:38.894735 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:38.894773 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:38.900008 20566 instancegroups.go:503] Cluster validated. I0803 22:17:38.900094 20566 instancegroups.go:400] Draining the node: "master-1b.local". I0803 22:17:38.901005 20566 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0803 22:17:38.901047 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:38.902229 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:38.902265 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:38.907591 20566 instancegroups.go:503] Cluster validated. I0803 22:17:38.907631 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:38.907651 20566 instancegroups.go:503] Cluster validated. I0803 22:17:38.907665 20566 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0803 22:17:38.908612 20566 instancegroups.go:400] Draining the node: "node-1a.local". I0803 22:17:38.909398 20566 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0803 22:17:38.909437 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:38.910655 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:38.910691 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:38.915962 20566 instancegroups.go:503] Cluster validated. I0803 22:17:38.916014 20566 instancegroups.go:400] Draining the node: "node-1b.local". I0803 22:17:38.916826 20566 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0803 22:17:38.916865 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:38.918076 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:38.918111 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:38.923384 20566 instancegroups.go:503] Cluster validated. I0803 22:17:38.923436 20566 instancegroups.go:400] Draining the node: "node-1c.local". I0803 22:17:38.924312 20566 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0803 22:17:38.924352 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:38.925526 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:38.925562 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:38.930852 20566 instancegroups.go:503] Cluster validated. I0803 22:17:38.930884 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:38.930904 20566 instancegroups.go:503] Cluster validated. I0803 22:17:38.930918 20566 instancegroups.go:311] Tainting 3 nodes in "node-2" instancegroup. I0803 22:17:38.931913 20566 instancegroups.go:400] Draining the node: "node-2a.local". I0803 22:17:38.932698 20566 instancegroups.go:591] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0803 22:17:38.932737 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:38.933958 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:38.933994 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:38.939244 20566 instancegroups.go:503] Cluster validated. I0803 22:17:38.939295 20566 instancegroups.go:400] Draining the node: "node-2b.local". I0803 22:17:38.940146 20566 instancegroups.go:591] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0803 22:17:38.940185 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:38.941381 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:38.941417 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:38.946697 20566 instancegroups.go:503] Cluster validated. I0803 22:17:38.946749 20566 instancegroups.go:400] Draining the node: "node-2c.local". I0803 22:17:38.947563 20566 instancegroups.go:591] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0803 22:17:38.947602 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:38.948825 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:38.948861 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:38.954181 20566 instancegroups.go:503] Cluster validated. I0803 22:17:38.954223 20566 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdate (0.07s) === RUN TestRollingUpdateAllNeedUpdateCloudonly I0803 22:17:38.954715 20566 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0803 22:17:38.954736 20566 instancegroups.go:436] waiting for 1ms after terminating instance W0803 22:17:38.955934 20566 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0803 22:17:38.956012 20566 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0803 22:17:38.956050 20566 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0803 22:17:38.956075 20566 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0803 22:17:38.956091 20566 instancegroups.go:436] waiting for 1ms after terminating instance W0803 22:17:38.957314 20566 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0803 22:17:38.957368 20566 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0803 22:17:38.957383 20566 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0803 22:17:38.957412 20566 instancegroups.go:436] waiting for 1ms after terminating instance W0803 22:17:38.958602 20566 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0803 22:17:38.958637 20566 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0803 22:17:38.958684 20566 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0803 22:17:38.958698 20566 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0803 22:17:38.958730 20566 instancegroups.go:436] waiting for 1ms after terminating instance W0803 22:17:38.959888 20566 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0803 22:17:38.959940 20566 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0803 22:17:38.959954 20566 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0803 22:17:38.959981 20566 instancegroups.go:436] waiting for 1ms after terminating instance W0803 22:17:38.961199 20566 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0803 22:17:38.961251 20566 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0803 22:17:38.961266 20566 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0803 22:17:38.961294 20566 instancegroups.go:436] waiting for 1ms after terminating instance W0803 22:17:38.962470 20566 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0803 22:17:38.962499 20566 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0803 22:17:38.962551 20566 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0803 22:17:38.962566 20566 instancegroups.go:591] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0803 22:17:38.962604 20566 instancegroups.go:436] waiting for 1ms after terminating instance W0803 22:17:38.963801 20566 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0803 22:17:38.963853 20566 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0803 22:17:38.963868 20566 instancegroups.go:591] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0803 22:17:38.963896 20566 instancegroups.go:436] waiting for 1ms after terminating instance W0803 22:17:38.965086 20566 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0803 22:17:38.965138 20566 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0803 22:17:38.965153 20566 instancegroups.go:591] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0803 22:17:38.965181 20566 instancegroups.go:436] waiting for 1ms after terminating instance W0803 22:17:38.966390 20566 instancegroups.go:468] Not validating cluster as cloudonly flag is set. I0803 22:17:38.966423 20566 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateCloudonly (0.01s) === RUN TestRollingUpdateAllNeedUpdateNoFailOnValidate I0803 22:17:38.966682 20566 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0803 22:17:38.966723 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:38.967921 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:38.967963 20566 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0803 22:17:38.967985 20566 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0803 22:17:38.968037 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:38.968062 20566 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0803 22:17:38.968087 20566 instancegroups.go:479] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0803 22:17:38.968095 20566 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0803 22:17:38.968925 20566 instancegroups.go:400] Draining the node: "master-1a.local". I0803 22:17:38.969682 20566 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0803 22:17:38.969721 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:38.970935 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:38.970980 20566 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0803 22:17:38.971003 20566 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0803 22:17:38.971042 20566 instancegroups.go:400] Draining the node: "master-1b.local". I0803 22:17:38.971910 20566 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0803 22:17:38.971948 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:38.973177 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:38.973218 20566 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0803 22:17:38.973240 20566 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0803 22:17:38.973261 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:38.973281 20566 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0803 22:17:38.973297 20566 instancegroups.go:479] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0803 22:17:38.973311 20566 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0803 22:17:38.974259 20566 instancegroups.go:400] Draining the node: "node-1a.local". I0803 22:17:38.975058 20566 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0803 22:17:38.975096 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:38.976323 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:38.976365 20566 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0803 22:17:38.976387 20566 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0803 22:17:38.976427 20566 instancegroups.go:400] Draining the node: "node-1b.local". I0803 22:17:38.977229 20566 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0803 22:17:38.977267 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:38.978469 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:38.978512 20566 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0803 22:17:38.978535 20566 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0803 22:17:38.978577 20566 instancegroups.go:400] Draining the node: "node-1c.local". I0803 22:17:38.979405 20566 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0803 22:17:38.979445 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:38.980657 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:38.980698 20566 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0803 22:17:38.980720 20566 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0803 22:17:38.980738 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:38.980757 20566 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0803 22:17:38.980771 20566 instancegroups.go:479] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0803 22:17:38.980787 20566 instancegroups.go:311] Tainting 3 nodes in "node-2" instancegroup. I0803 22:17:38.981787 20566 instancegroups.go:400] Draining the node: "node-2a.local". I0803 22:17:38.982655 20566 instancegroups.go:591] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0803 22:17:38.982694 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:38.983908 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:38.983949 20566 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0803 22:17:38.983971 20566 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0803 22:17:38.984014 20566 instancegroups.go:400] Draining the node: "node-2b.local". I0803 22:17:38.984824 20566 instancegroups.go:591] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0803 22:17:38.984862 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:38.986051 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:38.986093 20566 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0803 22:17:38.986115 20566 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0803 22:17:38.986159 20566 instancegroups.go:400] Draining the node: "node-2c.local". I0803 22:17:38.987014 20566 instancegroups.go:591] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0803 22:17:38.987054 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:38.988243 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:38.988286 20566 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0803 22:17:38.988310 20566 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0803 22:17:38.988331 20566 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateNoFailOnValidate (0.02s) === RUN TestRollingUpdateNoneNeedUpdate I0803 22:17:38.988568 20566 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdate (0.00s) === RUN TestRollingUpdateNoneNeedUpdateWithForce I0803 22:17:38.988855 20566 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0803 22:17:38.988891 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:38.990080 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:38.990115 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:38.995375 20566 instancegroups.go:503] Cluster validated. I0803 22:17:38.995456 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:38.995482 20566 instancegroups.go:503] Cluster validated. I0803 22:17:38.995495 20566 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0803 22:17:38.996253 20566 instancegroups.go:400] Draining the node: "master-1a.local". I0803 22:17:38.997009 20566 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0803 22:17:38.997048 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:38.998240 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:38.998276 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.003906 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.003964 20566 instancegroups.go:400] Draining the node: "master-1b.local". I0803 22:17:39.004834 20566 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0803 22:17:39.004874 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.006068 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.006111 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.011364 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.011404 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.011427 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.011440 20566 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0803 22:17:39.012434 20566 instancegroups.go:400] Draining the node: "node-1a.local". I0803 22:17:39.013198 20566 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0803 22:17:39.013237 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.014438 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.014474 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.019739 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.019791 20566 instancegroups.go:400] Draining the node: "node-1b.local". I0803 22:17:39.020624 20566 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0803 22:17:39.020664 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.021859 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.021901 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.027155 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.027208 20566 instancegroups.go:400] Draining the node: "node-1c.local". I0803 22:17:39.028061 20566 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0803 22:17:39.028101 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.029296 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.029332 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.034570 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.034603 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.034623 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.034639 20566 instancegroups.go:311] Tainting 3 nodes in "node-2" instancegroup. I0803 22:17:39.035620 20566 instancegroups.go:400] Draining the node: "node-2a.local". I0803 22:17:39.036412 20566 instancegroups.go:591] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0803 22:17:39.036452 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.037650 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.037685 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.042954 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.043011 20566 instancegroups.go:400] Draining the node: "node-2b.local". I0803 22:17:39.043833 20566 instancegroups.go:591] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0803 22:17:39.043872 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.045064 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.045100 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.050341 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.050398 20566 instancegroups.go:400] Draining the node: "node-2c.local". I0803 22:17:39.051235 20566 instancegroups.go:591] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0803 22:17:39.051275 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.052459 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.052495 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.057760 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.057794 20566 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdateWithForce (0.07s) === RUN TestRollingUpdateEmptyGroup I0803 22:17:39.057924 20566 rollingupdate.go:96] Cloud Instance Group length is zero. Not doing a rolling-update. --- PASS: TestRollingUpdateEmptyGroup (0.00s) === RUN TestRollingUpdateUnknownRole --- PASS: TestRollingUpdateUnknownRole (0.00s) === RUN TestRollingUpdateAllNeedUpdateFailsValidation I0803 22:17:39.058390 20566 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0803 22:17:39.058422 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.059623 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.059666 20566 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. E0803 22:17:39.059687 20566 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateFailsValidation (0.00s) === RUN TestRollingUpdateAllNeedUpdateErrorsValidation I0803 22:17:39.059999 20566 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0803 22:17:39.060032 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.061226 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.061269 20566 instancegroups.go:513] Cluster did not validate within deadline: testing validation error. E0803 22:17:39.061294 20566 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateFailsValidation I0803 22:17:39.061523 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.061553 20566 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. E0803 22:17:39.061572 20566 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateNodes1NeedsUpdateFailsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateErrorsValidation I0803 22:17:39.061719 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.061744 20566 instancegroups.go:513] Cluster did not validate within deadline: testing validation error. E0803 22:17:39.061783 20566 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateNodes1NeedsUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneMaster I0803 22:17:39.062026 20566 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0803 22:17:39.062059 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.063254 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.063303 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.068611 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.068680 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.068708 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.068723 20566 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0803 22:17:39.069459 20566 instancegroups.go:400] Draining the node: "master-1a.local". I0803 22:17:39.070221 20566 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0803 22:17:39.070260 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.071464 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.071512 20566 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. E0803 22:17:39.071536 20566 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneMaster I0803 22:17:39.071839 20566 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0803 22:17:39.071872 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.073072 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.073114 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.078369 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.078436 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.078463 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.078476 20566 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0803 22:17:39.079266 20566 instancegroups.go:400] Draining the node: "master-1a.local". I0803 22:17:39.080025 20566 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0803 22:17:39.080064 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.081285 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.081328 20566 instancegroups.go:513] Cluster did not validate within deadline: testing validation error. E0803 22:17:39.081351 20566 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateNonRelatedInstanceGroupFailure I0803 22:17:39.081617 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.081644 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.081657 20566 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0803 22:17:39.082636 20566 instancegroups.go:400] Draining the node: "node-1a.local". I0803 22:17:39.083416 20566 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0803 22:17:39.083454 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.084660 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.084697 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.089926 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.089977 20566 instancegroups.go:400] Draining the node: "node-1b.local". I0803 22:17:39.090776 20566 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0803 22:17:39.090816 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.092023 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.092060 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.097324 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.097375 20566 instancegroups.go:400] Draining the node: "node-1c.local". I0803 22:17:39.098188 20566 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0803 22:17:39.098227 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.099437 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.099474 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.104766 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.104801 20566 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNonRelatedInstanceGroupFailure (0.02s) === RUN TestRollingUpdateRelatedInstanceGroupFailure I0803 22:17:39.105063 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.105094 20566 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. E0803 22:17:39.105112 20566 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateRelatedInstanceGroupFailure (0.00s) === RUN TestRollingUpdateMasterGroupFailure I0803 22:17:39.105377 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.105406 20566 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. E0803 22:17:39.105421 20566 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateMasterGroupFailure (0.00s) === RUN TestRollingUpdateValidationErrorInstanceGroupNil I0803 22:17:39.105664 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.105692 20566 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. E0803 22:17:39.105708 20566 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateValidationErrorInstanceGroupNil (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneNode I0803 22:17:39.105871 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.105899 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.105912 20566 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0803 22:17:39.106873 20566 instancegroups.go:400] Draining the node: "node-1a.local". I0803 22:17:39.107607 20566 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0803 22:17:39.107648 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.108843 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.108891 20566 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. E0803 22:17:39.108913 20566 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneNode I0803 22:17:39.109127 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.109158 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.109172 20566 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0803 22:17:39.110149 20566 instancegroups.go:400] Draining the node: "node-1a.local". I0803 22:17:39.110870 20566 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0803 22:17:39.110907 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.112172 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.112217 20566 instancegroups.go:513] Cluster did not validate within deadline: testing validation error. E0803 22:17:39.112240 20566 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateFlappingValidation I0803 22:17:39.112537 20566 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0803 22:17:39.112570 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.113776 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.113824 20566 instancegroups.go:526] Cluster did not pass validation, will retry in "1ms": testing failure. I0803 22:17:39.114993 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.120269 20566 instancegroups.go:526] Cluster did not pass validation, will retry in "1ms": testing failure. I0803 22:17:39.121467 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.126715 20566 instancegroups.go:526] Cluster did not pass validation, will retry in "1ms": testing failure. I0803 22:17:39.127888 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.133061 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.133139 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.133165 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.133181 20566 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0803 22:17:39.133922 20566 instancegroups.go:400] Draining the node: "master-1a.local". I0803 22:17:39.134669 20566 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0803 22:17:39.134708 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.135894 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.135937 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.141239 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.141304 20566 instancegroups.go:400] Draining the node: "master-1b.local". I0803 22:17:39.142154 20566 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0803 22:17:39.142193 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.143403 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.143460 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.148810 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.148850 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.148878 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.148897 20566 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0803 22:17:39.149827 20566 instancegroups.go:400] Draining the node: "node-1a.local". I0803 22:17:39.150576 20566 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0803 22:17:39.150614 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.151804 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.151854 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.157141 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.157201 20566 instancegroups.go:400] Draining the node: "node-1b.local". I0803 22:17:39.158000 20566 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0803 22:17:39.158039 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.159222 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.159280 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.164632 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.164693 20566 instancegroups.go:400] Draining the node: "node-1c.local". I0803 22:17:39.165532 20566 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0803 22:17:39.165571 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.166781 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.166831 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.172150 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.172183 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.172210 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.172225 20566 instancegroups.go:311] Tainting 3 nodes in "node-2" instancegroup. I0803 22:17:39.173157 20566 instancegroups.go:400] Draining the node: "node-2a.local". I0803 22:17:39.173943 20566 instancegroups.go:591] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0803 22:17:39.173982 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.175171 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.175243 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.180511 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.180569 20566 instancegroups.go:400] Draining the node: "node-2b.local". I0803 22:17:39.181553 20566 instancegroups.go:591] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0803 22:17:39.181593 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.182811 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.182862 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.188214 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.188273 20566 instancegroups.go:400] Draining the node: "node-2c.local". I0803 22:17:39.189233 20566 instancegroups.go:591] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0803 22:17:39.189285 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.190498 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.190548 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.195830 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.195865 20566 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateFlappingValidation (0.08s) === RUN TestRollingUpdateValidatesAfterBastion I0803 22:17:39.196219 20566 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0803 22:17:39.196252 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.197445 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.197496 20566 instancegroups.go:526] Cluster did not pass validation, will retry in "1ms": testing failure. I0803 22:17:39.198662 20566 instancegroups.go:526] Cluster did not pass validation, will retry in "1ms": testing failure. I0803 22:17:39.199782 20566 instancegroups.go:526] Cluster did not pass validation, will retry in "1ms": testing failure. I0803 22:17:39.200887 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.206154 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.206218 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.206249 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.206272 20566 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0803 22:17:39.207006 20566 instancegroups.go:400] Draining the node: "master-1a.local". I0803 22:17:39.207772 20566 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0803 22:17:39.207810 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.208991 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.209029 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.214324 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.214378 20566 instancegroups.go:400] Draining the node: "master-1b.local". I0803 22:17:39.215189 20566 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0803 22:17:39.215229 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.216437 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.216474 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.221732 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.221773 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.221795 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.221810 20566 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0803 22:17:39.222816 20566 instancegroups.go:400] Draining the node: "node-1a.local". I0803 22:17:39.223619 20566 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0803 22:17:39.223658 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.224864 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.224901 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.230201 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.230254 20566 instancegroups.go:400] Draining the node: "node-1b.local". I0803 22:17:39.231081 20566 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0803 22:17:39.231119 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.232297 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.232335 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.237627 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.237681 20566 instancegroups.go:400] Draining the node: "node-1c.local". I0803 22:17:39.238490 20566 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0803 22:17:39.238528 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.239741 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.239778 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.245070 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.245103 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.245125 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.245139 20566 instancegroups.go:311] Tainting 3 nodes in "node-2" instancegroup. I0803 22:17:39.246176 20566 instancegroups.go:400] Draining the node: "node-2a.local". I0803 22:17:39.247083 20566 instancegroups.go:591] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0803 22:17:39.247122 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.248328 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.248365 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.253980 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.254035 20566 instancegroups.go:400] Draining the node: "node-2b.local". I0803 22:17:39.255019 20566 instancegroups.go:591] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0803 22:17:39.255059 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.256267 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.256304 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.261598 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.261651 20566 instancegroups.go:400] Draining the node: "node-2c.local". I0803 22:17:39.262681 20566 instancegroups.go:591] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0803 22:17:39.262721 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.263936 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.263972 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.269220 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.269255 20566 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateValidatesAfterBastion (0.07s) === RUN TestAddAnnotatedNodesToNeedsUpdate --- PASS: TestAddAnnotatedNodesToNeedsUpdate (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateCloudonly --- PASS: TestAddAnnotatedNodesToNeedsUpdateCloudonly (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateNodesMissing --- PASS: TestAddAnnotatedNodesToNeedsUpdateNodesMissing (0.00s) === RUN TestRollingUpdateTaintAllButOneNeedUpdate I0803 22:17:39.269961 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.269990 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.270003 20566 instancegroups.go:311] Tainting 2 nodes in "node-1" instancegroup. I0803 22:17:39.270862 20566 instancegroups.go:400] Draining the node: "node-1a.local". I0803 22:17:39.271849 20566 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0803 22:17:39.271886 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.273103 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.273139 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.278427 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.278478 20566 instancegroups.go:400] Draining the node: "node-1b.local". I0803 22:17:39.279495 20566 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0803 22:17:39.279535 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.280748 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.280789 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.286102 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.286136 20566 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateTaintAllButOneNeedUpdate (0.02s) === RUN TestRollingUpdateMaxSurgeIgnoredForMaster I0803 22:17:39.286421 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.286450 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.286462 20566 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0803 22:17:39.287337 20566 instancegroups.go:400] Draining the node: "master-1a.local". I0803 22:17:39.288280 20566 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0803 22:17:39.288318 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.289533 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.289572 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.294892 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.294943 20566 instancegroups.go:400] Draining the node: "master-1b.local". I0803 22:17:39.295930 20566 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0803 22:17:39.295970 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.297199 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.297234 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.302532 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.302567 20566 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeIgnoredForMaster (0.02s) === RUN TestRollingUpdateDisabled I0803 22:17:39.302927 20566 instancegroups.go:185] Rolling updates for InstanceGroup bastion-1 are disabled W0803 22:17:39.302968 20566 instancegroups.go:468] Not validating cluster as cloudonly flag is set. I0803 22:17:39.302989 20566 instancegroups.go:185] Rolling updates for InstanceGroup master-1 are disabled W0803 22:17:39.303003 20566 instancegroups.go:468] Not validating cluster as cloudonly flag is set. I0803 22:17:39.303022 20566 instancegroups.go:185] Rolling updates for InstanceGroup node-1 are disabled W0803 22:17:39.303031 20566 instancegroups.go:468] Not validating cluster as cloudonly flag is set. I0803 22:17:39.303049 20566 instancegroups.go:185] Rolling updates for InstanceGroup node-2 are disabled I0803 22:17:39.303061 20566 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabled (0.00s) === RUN TestRollingUpdateDisabledSurge I0803 22:17:39.303379 20566 instancegroups.go:570] Detaching instance "bastion-1a", in group "bastion-1". I0803 22:17:39.303437 20566 instancegroups.go:172] waiting for 1ms after detaching instance I0803 22:17:39.304638 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.304674 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.309962 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.309999 20566 instancegroups.go:185] Rolling updates for InstanceGroup bastion-1 are disabled I0803 22:17:39.310051 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.310072 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.310085 20566 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0803 22:17:39.310905 20566 instancegroups.go:185] Rolling updates for InstanceGroup master-1 are disabled I0803 22:17:39.310937 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.310957 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.310969 20566 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0803 22:17:39.312748 20566 instancegroups.go:568] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0803 22:17:39.312815 20566 instancegroups.go:172] waiting for 1ms after detaching instance I0803 22:17:39.313965 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.314352 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.319825 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.319868 20566 instancegroups.go:185] Rolling updates for InstanceGroup node-1 are disabled I0803 22:17:39.319888 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.319908 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.319925 20566 instancegroups.go:311] Tainting 3 nodes in "node-2" instancegroup. I0803 22:17:39.321009 20566 instancegroups.go:568] Detaching instance "node-2c", node "node-2c.local", in group "node-2". I0803 22:17:39.321080 20566 instancegroups.go:172] waiting for 1ms after detaching instance I0803 22:17:39.322272 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.322307 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:39.327523 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.327560 20566 instancegroups.go:185] Rolling updates for InstanceGroup node-2 are disabled I0803 22:17:39.327579 20566 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabledSurge (0.02s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdate I0803 22:17:39.327880 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.327916 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.327930 20566 instancegroups.go:311] Tainting 7 nodes in "node-1" instancegroup. I0803 22:17:39.330154 20566 instancegroups.go:400] Draining the node: "node-1a.local". I0803 22:17:39.330954 20566 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0803 22:17:39.331018 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.332191 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.332237 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.332282 20566 instancegroups.go:400] Draining the node: "node-1c.local". I0803 22:17:39.332352 20566 instancegroups.go:400] Draining the node: "node-1b.local". I0803 22:17:39.333681 20566 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0803 22:17:39.333975 20566 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0803 22:17:39.334017 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.335225 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.335296 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.435834 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.435880 20566 instancegroups.go:400] Draining the node: "node-1e.local". I0803 22:17:39.435963 20566 instancegroups.go:400] Draining the node: "node-1d.local". I0803 22:17:39.437277 20566 instancegroups.go:591] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0803 22:17:39.437584 20566 instancegroups.go:591] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0803 22:17:39.437627 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.438832 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.438877 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.439037 20566 instancegroups.go:400] Draining the node: "node-1f.local". I0803 22:17:39.439990 20566 instancegroups.go:591] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0803 22:17:39.440034 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.441241 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.441309 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.541815 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.541929 20566 instancegroups.go:400] Draining the node: "node-1g.local". I0803 22:17:39.543637 20566 instancegroups.go:591] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0803 22:17:39.543722 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.544967 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.545016 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.545035 20566 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxUnavailableAllButOneNeedUpdate I0803 22:17:39.545302 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.545336 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.545351 20566 instancegroups.go:311] Tainting 6 nodes in "node-1" instancegroup. I0803 22:17:39.547233 20566 instancegroups.go:400] Draining the node: "node-1b.local". I0803 22:17:39.547303 20566 instancegroups.go:400] Draining the node: "node-1a.local". I0803 22:17:39.548575 20566 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0803 22:17:39.548948 20566 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0803 22:17:39.548995 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.550170 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.550259 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.650855 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.650933 20566 instancegroups.go:400] Draining the node: "node-1d.local". I0803 22:17:39.650954 20566 instancegroups.go:400] Draining the node: "node-1c.local". I0803 22:17:39.652693 20566 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0803 22:17:39.652981 20566 instancegroups.go:591] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0803 22:17:39.653024 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.654198 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.654247 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.654291 20566 instancegroups.go:400] Draining the node: "node-1e.local". I0803 22:17:39.655146 20566 instancegroups.go:591] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0803 22:17:39.655189 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.656361 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.656437 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.756893 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.756968 20566 instancegroups.go:400] Draining the node: "node-1f.local". I0803 22:17:39.758074 20566 instancegroups.go:591] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0803 22:17:39.758125 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.759318 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.759364 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.759384 20566 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllButOneNeedUpdate (0.21s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdateMaster I0803 22:17:39.759713 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.759749 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.759764 20566 instancegroups.go:311] Tainting 7 nodes in "master-1" instancegroup. I0803 22:17:39.762285 20566 instancegroups.go:400] Draining the node: "master-1a.local". I0803 22:17:39.763101 20566 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0803 22:17:39.763150 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.764346 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.764391 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.764437 20566 instancegroups.go:400] Draining the node: "master-1c.local". I0803 22:17:39.764493 20566 instancegroups.go:400] Draining the node: "master-1b.local". I0803 22:17:39.765796 20566 instancegroups.go:591] Stopping instance "master-1c", node "master-1c.local", in group "master-1" (this may take a while). I0803 22:17:39.766080 20566 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0803 22:17:39.766123 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.767328 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.767409 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.867986 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.868104 20566 instancegroups.go:400] Draining the node: "master-1e.local". I0803 22:17:39.868156 20566 instancegroups.go:400] Draining the node: "master-1d.local". I0803 22:17:39.869685 20566 instancegroups.go:591] Stopping instance "master-1d", node "master-1d.local", in group "master-1" (this may take a while). I0803 22:17:39.870147 20566 instancegroups.go:591] Stopping instance "master-1e", node "master-1e.local", in group "master-1" (this may take a while). I0803 22:17:39.870218 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.871422 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.871469 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.871511 20566 instancegroups.go:400] Draining the node: "master-1f.local". I0803 22:17:39.872395 20566 instancegroups.go:591] Stopping instance "master-1f", node "master-1f.local", in group "master-1" (this may take a while). I0803 22:17:39.872442 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.873610 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.873699 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.974317 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.974379 20566 instancegroups.go:400] Draining the node: "master-1g.local". I0803 22:17:39.975248 20566 instancegroups.go:591] Stopping instance "master-1g", node "master-1g.local", in group "master-1" (this may take a while). I0803 22:17:39.975295 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.976511 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.976557 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.976577 20566 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdateMaster (0.22s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdate I0803 22:17:39.976869 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.976905 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.976919 20566 instancegroups.go:311] Tainting 6 nodes in "node-1" instancegroup. I0803 22:17:39.978748 20566 instancegroups.go:568] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0803 22:17:39.978841 20566 instancegroups.go:172] waiting for 1ms after detaching instance I0803 22:17:39.980048 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.980094 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.980108 20566 instancegroups.go:568] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0803 22:17:39.980165 20566 instancegroups.go:172] waiting for 1ms after detaching instance I0803 22:17:39.981375 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.981418 20566 instancegroups.go:503] Cluster validated. I0803 22:17:39.981466 20566 instancegroups.go:400] Draining the node: "node-1b.local". I0803 22:17:39.981542 20566 instancegroups.go:400] Draining the node: "node-1a.local". I0803 22:17:39.982863 20566 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0803 22:17:39.983154 20566 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0803 22:17:39.983197 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:39.984371 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:39.984446 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.084877 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.084961 20566 instancegroups.go:400] Draining the node: "node-1d.local". I0803 22:17:40.084975 20566 instancegroups.go:400] Draining the node: "node-1c.local". I0803 22:17:40.086479 20566 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0803 22:17:40.086781 20566 instancegroups.go:591] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0803 22:17:40.086830 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.088038 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.088085 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.088127 20566 instancegroups.go:400] Draining the node: "node-1e.local". I0803 22:17:40.089024 20566 instancegroups.go:591] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0803 22:17:40.089074 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.090275 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.090346 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.190944 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.191020 20566 instancegroups.go:400] Draining the node: "node-1f.local". I0803 22:17:40.191969 20566 instancegroups.go:591] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0803 22:17:40.192029 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.193234 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.193282 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.193302 20566 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeAllButOneNeedUpdate I0803 22:17:40.193596 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.193631 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.193646 20566 instancegroups.go:311] Tainting 6 nodes in "node-1" instancegroup. I0803 22:17:40.195481 20566 instancegroups.go:568] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0803 22:17:40.195541 20566 instancegroups.go:568] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0803 22:17:40.195601 20566 instancegroups.go:172] waiting for 1ms after detaching instance I0803 22:17:40.196816 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.196863 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.196909 20566 instancegroups.go:400] Draining the node: "node-1b.local". I0803 22:17:40.196977 20566 instancegroups.go:400] Draining the node: "node-1a.local". I0803 22:17:40.198272 20566 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0803 22:17:40.198582 20566 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0803 22:17:40.198632 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.199815 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.199897 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.300381 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.300467 20566 instancegroups.go:400] Draining the node: "node-1d.local". I0803 22:17:40.300504 20566 instancegroups.go:400] Draining the node: "node-1c.local". I0803 22:17:40.301996 20566 instancegroups.go:591] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0803 22:17:40.302350 20566 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0803 22:17:40.302419 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.303629 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.303677 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.303731 20566 instancegroups.go:400] Draining the node: "node-1e.local". I0803 22:17:40.304666 20566 instancegroups.go:591] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0803 22:17:40.304719 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.305909 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.305981 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.406728 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.406794 20566 instancegroups.go:400] Draining the node: "node-1f.local". I0803 22:17:40.407665 20566 instancegroups.go:591] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0803 22:17:40.407717 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.408844 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.408894 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.408914 20566 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllButOneNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeGreaterThanNeedUpdate I0803 22:17:40.409185 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.409212 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.409225 20566 instancegroups.go:311] Tainting 2 nodes in "node-1" instancegroup. I0803 22:17:40.409902 20566 instancegroups.go:568] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0803 22:17:40.409936 20566 instancegroups.go:568] Detaching instance "node-1a", node "node-1a.local", in group "node-1". I0803 22:17:40.409974 20566 instancegroups.go:172] waiting for 1ms after detaching instance I0803 22:17:40.411143 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.411184 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:40.416446 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.416514 20566 instancegroups.go:400] Draining the node: "node-1b.local". I0803 22:17:40.416580 20566 instancegroups.go:400] Draining the node: "node-1a.local". I0803 22:17:40.417845 20566 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0803 22:17:40.417884 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.418173 20566 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0803 22:17:40.418208 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.419051 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.419092 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:40.424407 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.424446 20566 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeGreaterThanNeedUpdate (0.02s) === RUN TestRollingUpdateDetachFails I0803 22:17:40.424661 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.424690 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.424702 20566 instancegroups.go:311] Tainting 2 nodes in "node-1" instancegroup. I0803 22:17:40.425408 20566 instancegroups.go:568] Detaching instance "node-1b", node "node-1b.local", in group "node-1". E0803 22:17:40.425491 20566 instancegroups.go:160] Failed to detach instance "node-1b": error detaching instance "node-1b", node "node-1b.local": error detaching instance "node-1b": testing error I0803 22:17:40.425509 20566 instancegroups.go:568] Detaching instance "node-1a", node "node-1a.local", in group "node-1". E0803 22:17:40.425551 20566 instancegroups.go:160] Failed to detach instance "node-1a": error detaching instance "node-1a", node "node-1a.local": error detaching instance "node-1a": testing error I0803 22:17:40.425585 20566 instancegroups.go:172] waiting for 1ms after detaching instance I0803 22:17:40.426749 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.426790 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:40.433009 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.433137 20566 instancegroups.go:400] Draining the node: "node-1b.local". I0803 22:17:40.433156 20566 instancegroups.go:400] Draining the node: "node-1a.local". I0803 22:17:40.434552 20566 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0803 22:17:40.434590 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.434888 20566 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0803 22:17:40.434939 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.435791 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.435827 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:40.441378 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.441417 20566 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDetachFails (0.02s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached I0803 22:17:40.441726 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.441768 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.441783 20566 instancegroups.go:311] Tainting 4 nodes in "node-1" instancegroup. I0803 22:17:40.443086 20566 instancegroups.go:568] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0803 22:17:40.443155 20566 instancegroups.go:172] waiting for 1ms after detaching instance I0803 22:17:40.444373 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.444419 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.444434 20566 instancegroups.go:568] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0803 22:17:40.444492 20566 instancegroups.go:172] waiting for 1ms after detaching instance I0803 22:17:40.445681 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.445728 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.445778 20566 instancegroups.go:400] Draining the node: "node-1c.local". I0803 22:17:40.445854 20566 instancegroups.go:400] Draining the node: "node-1a.local". I0803 22:17:40.445917 20566 instancegroups.go:400] Draining the node: "node-1b.local". I0803 22:17:40.447550 20566 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0803 22:17:40.447607 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.447951 20566 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0803 22:17:40.448004 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.448268 20566 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0803 22:17:40.448309 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.448836 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.549362 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.549430 20566 instancegroups.go:400] Draining the node: "node-1d.local". I0803 22:17:40.550321 20566 instancegroups.go:591] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0803 22:17:40.550374 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.551592 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.551637 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.551655 20566 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached (0.11s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached I0803 22:17:40.551897 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.551929 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.551943 20566 instancegroups.go:311] Tainting 7 nodes in "node-1" instancegroup. I0803 22:17:40.554067 20566 instancegroups.go:400] Draining the node: "node-1a.local". I0803 22:17:40.554879 20566 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0803 22:17:40.554925 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.556119 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.556164 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.556209 20566 instancegroups.go:400] Draining the node: "node-1e.local". I0803 22:17:40.556282 20566 instancegroups.go:400] Draining the node: "node-1c.local". I0803 22:17:40.557679 20566 instancegroups.go:591] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0803 22:17:40.558035 20566 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0803 22:17:40.558089 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.559283 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.559356 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.660226 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.660300 20566 instancegroups.go:400] Draining the node: "node-1g.local". I0803 22:17:40.660374 20566 instancegroups.go:400] Draining the node: "node-1f.local". I0803 22:17:40.661814 20566 instancegroups.go:591] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0803 22:17:40.662211 20566 instancegroups.go:591] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0803 22:17:40.662269 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.663471 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.663516 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.663558 20566 instancegroups.go:400] Draining the node: "node-1b.local". I0803 22:17:40.664392 20566 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0803 22:17:40.664437 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.665580 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.665652 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.766259 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.766335 20566 instancegroups.go:400] Draining the node: "node-1d.local". I0803 22:17:40.767280 20566 instancegroups.go:591] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0803 22:17:40.767332 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.768485 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.768532 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.768551 20566 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached (0.22s) === RUN TestRollingUpdateOnlyWarmPoolNodes I0803 22:17:40.768840 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.768869 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.768890 20566 instancegroups.go:132] deleting warm pool instance "node-1-wp-a" I0803 22:17:40.768913 20566 instancegroups.go:132] deleting warm pool instance "node-1-wp-b" I0803 22:17:40.768933 20566 instancegroups.go:132] deleting warm pool instance "node-1-wp-c" I0803 22:17:40.768951 20566 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateOnlyWarmPoolNodes (0.00s) === RUN TestRollingWarmPoolBeforeJoinedNodes I0803 22:17:40.769098 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.769122 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.769135 20566 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0803 22:17:40.770124 20566 instancegroups.go:132] deleting warm pool instance "node-1-wp-a" I0803 22:17:40.770157 20566 instancegroups.go:132] deleting warm pool instance "node-1-wp-b" I0803 22:17:40.770177 20566 instancegroups.go:132] deleting warm pool instance "node-1-wp-c" I0803 22:17:40.770220 20566 instancegroups.go:400] Draining the node: "node-1a.local". I0803 22:17:40.770986 20566 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0803 22:17:40.771021 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.772178 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.772214 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:40.777483 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.777533 20566 instancegroups.go:400] Draining the node: "node-1b.local". I0803 22:17:40.778373 20566 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0803 22:17:40.778413 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.779619 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.779654 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:40.784900 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.784951 20566 instancegroups.go:400] Draining the node: "node-1c.local". I0803 22:17:40.785916 20566 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0803 22:17:40.785970 20566 instancegroups.go:436] waiting for 1ms after terminating instance I0803 22:17:40.787180 20566 instancegroups.go:470] Validating the cluster. I0803 22:17:40.787223 20566 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0803 22:17:40.792530 20566 instancegroups.go:503] Cluster validated. I0803 22:17:40.792564 20566 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingWarmPoolBeforeJoinedNodes (0.02s) === RUN TestSettings === RUN TestSettings/DrainAndTerminate === RUN TestSettings/MaxUnavailable === RUN TestSettings/MaxSurge --- PASS: TestSettings (0.00s) --- PASS: TestSettings/DrainAndTerminate (0.00s) --- PASS: TestSettings/MaxUnavailable (0.00s) --- PASS: TestSettings/MaxSurge (0.00s) === RUN TestMaxUnavailable === RUN TestMaxUnavailable/0_1 === RUN TestMaxUnavailable/0%_1 === RUN TestMaxUnavailable/39%_10 === RUN TestMaxUnavailable/100%_10 --- PASS: TestMaxUnavailable (0.00s) --- PASS: TestMaxUnavailable/0_1 (0.00s) --- PASS: TestMaxUnavailable/0%_1 (0.00s) --- PASS: TestMaxUnavailable/39%_10 (0.00s) --- PASS: TestMaxUnavailable/100%_10 (0.00s) === RUN TestMaxSurge === RUN TestMaxSurge/0_1 === RUN TestMaxSurge/0%_1 === RUN TestMaxSurge/31%_10 === RUN TestMaxSurge/100%_10 --- PASS: TestMaxSurge (0.00s) --- PASS: TestMaxSurge/0_1 (0.00s) --- PASS: TestMaxSurge/0%_1 (0.00s) --- PASS: TestMaxSurge/31%_10 (0.00s) --- PASS: TestMaxSurge/100%_10 (0.00s) === RUN TestAWSDefault --- PASS: TestAWSDefault (0.00s) PASS ok k8s.io/kops/pkg/instancegroups 1.979s === RUN TestWriteToken === RUN TestWriteToken/WriteJson === RUN TestWriteToken/WriteJsonWithIndent --- PASS: TestWriteToken (0.00s) --- PASS: TestWriteToken/WriteJson (0.00s) --- PASS: TestWriteToken/WriteJsonWithIndent (0.00s) PASS ok k8s.io/kops/pkg/jsonutils 0.047s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.00s) PASS ok k8s.io/kops/pkg/k8scodecs 0.013s === RUN TestParse E0803 22:16:43.838792 6566 versions.go:96] unable to parse Kubernetes version "" E0803 22:16:43.838875 6566 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: TestParse (0.00s) === RUN TestIsGTE === RUN TestIsGTE/KV_greater_than_Version === RUN TestIsGTE/KV_greater_than_Version#01 === RUN TestIsGTE/KV_equal_Version === RUN TestIsGTE/KV_equal_Version#01 === RUN TestIsGTE/Version_greater_than_KV === RUN TestIsGTE/Version_greater_than_KV#01 --- PASS: TestIsGTE (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version#01 (0.00s) --- PASS: TestIsGTE/KV_equal_Version (0.00s) --- PASS: TestIsGTE/KV_equal_Version#01 (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV#01 (0.00s) PASS ok k8s.io/kops/pkg/k8sversion 0.046s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.01s) === RUN TestToVersionedJSON --- PASS: TestToVersionedJSON (0.00s) === RUN TestRewriteAPIGroup --- PASS: TestRewriteAPIGroup (0.00s) PASS ok k8s.io/kops/pkg/kopscodecs 0.052s === RUN TestBuildKubecfg === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster === RUN TestBuildKubecfg/Public_DNS_with_kops_auth_plugin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port --- PASS: TestBuildKubecfg (0.15s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster (0.00s) --- PASS: TestBuildKubecfg/Public_DNS_with_kops_auth_plugin (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port (0.02s) PASS ok k8s.io/kops/pkg/kubeconfig 0.209s ? k8s.io/kops/pkg/kubemanifest [no test files] === RUN Test_ProxyFunc --- PASS: Test_ProxyFunc (0.00s) === RUN TestBootstrapUserData --- PASS: TestBootstrapUserData (0.01s) === RUN TestValidateAWSVolumeAllow50ratio --- PASS: TestValidateAWSVolumeAllow50ratio (0.00s) PASS ok k8s.io/kops/pkg/model 0.080s === RUN TestRootVolumeOptimizationFlag --- PASS: TestRootVolumeOptimizationFlag (0.00s) === RUN TestAPIServerAdditionalSecurityGroupsWithNLB === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Master === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Node --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Master (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Node (0.00s) === RUN TestJoinSuffixes --- PASS: TestJoinSuffixes (0.00s) === RUN TestIAMServiceEC2 --- PASS: TestIAMServiceEC2 (0.00s) === RUN Test_formatAWSIAMStatement === RUN Test_formatAWSIAMStatement/namespace_and_name_without_wildcard === RUN Test_formatAWSIAMStatement/name_contains_wildcard === RUN Test_formatAWSIAMStatement/namespace_contains_wildcard --- PASS: Test_formatAWSIAMStatement (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_and_name_without_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/name_contains_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_contains_wildcard (0.00s) PASS ok k8s.io/kops/pkg/model/awsmodel 0.058s === RUN TestAPILoadBalancerModelBuilder_Build --- PASS: TestAPILoadBalancerModelBuilder_Build (0.00s) === RUN TestSubnetForLoadbalancer --- PASS: TestSubnetForLoadbalancer (0.00s) === RUN TestCloudTagsForInstanceGroup --- PASS: TestCloudTagsForInstanceGroup (0.00s) === RUN TestNetworkModelBuilder_Build --- PASS: TestNetworkModelBuilder_Build (0.00s) === RUN TestResourceGroupModelBuilder_Build --- PASS: TestResourceGroupModelBuilder_Build (0.00s) === RUN TestVMScaleSetModelBuilder_Build --- PASS: TestVMScaleSetModelBuilder_Build (0.00s) === RUN TestGetCapacity === RUN TestGetCapacity/test_case_0 === RUN TestGetCapacity/test_case_1 === RUN TestGetCapacity/test_case_2 === RUN TestGetCapacity/test_case_3 --- PASS: TestGetCapacity (0.00s) --- PASS: TestGetCapacity/test_case_0 (0.00s) --- PASS: TestGetCapacity/test_case_1 (0.00s) --- PASS: TestGetCapacity/test_case_2 (0.00s) --- PASS: TestGetCapacity/test_case_3 (0.00s) === RUN TestGetStorageProfile === RUN TestGetStorageProfile/test_case_0 === RUN TestGetStorageProfile/test_case_1 --- PASS: TestGetStorageProfile (0.00s) --- PASS: TestGetStorageProfile/test_case_0 (0.00s) --- PASS: TestGetStorageProfile/test_case_1 (0.00s) === RUN TestParseImage === RUN TestParseImage/test_case_0 === RUN TestParseImage/test_case_1 === RUN TestParseImage/test_case_2 === RUN TestParseImage/test_case_3 --- PASS: TestParseImage (0.00s) --- PASS: TestParseImage/test_case_0 (0.00s) --- PASS: TestParseImage/test_case_1 (0.00s) --- PASS: TestParseImage/test_case_2 (0.00s) --- PASS: TestParseImage/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/model/azuremodel 0.076s === RUN TestCloudConfigurationOptionsBuilder === RUN TestCloudConfigurationOptionsBuilder/neither === RUN TestCloudConfigurationOptionsBuilder/all_false === RUN TestCloudConfigurationOptionsBuilder/all_true === RUN TestCloudConfigurationOptionsBuilder/os_false === RUN TestCloudConfigurationOptionsBuilder/os_true === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_true === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_true --- PASS: TestCloudConfigurationOptionsBuilder (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/neither (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_true (0.00s) === RUN Test_Build_Containerd_Supported_Version --- PASS: Test_Build_Containerd_Supported_Version (0.00s) === RUN Test_Build_Containerd_Unneeded_Runtime --- PASS: Test_Build_Containerd_Unneeded_Runtime (0.00s) === RUN Test_Build_Containerd_Needed_Runtime --- PASS: Test_Build_Containerd_Needed_Runtime (0.00s) === RUN TestImage I0803 22:17:14.331710 18671 featureflag.go:162] FeatureFlag "ImageDigest"=false --- PASS: TestImage (0.00s) === RUN Test_Build_KCM_Builder --- PASS: Test_Build_KCM_Builder (0.00s) === RUN Test_Build_KCM_Builder_Change_Duration --- PASS: Test_Build_KCM_Builder_Change_Duration (0.00s) === RUN Test_Build_KCM_Builder_CIDR_Mask_Size === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: (0.00s) === RUN TestFeatureGatesKubernetesVersion --- PASS: TestFeatureGatesKubernetesVersion (0.00s) === RUN TestFeatureGatesOverride --- PASS: TestFeatureGatesOverride (0.00s) === RUN Test_Build_Scheduler_Without_PolicyConfigMap --- PASS: Test_Build_Scheduler_Without_PolicyConfigMap (0.00s) === RUN Test_Build_Scheduler_PolicyConfigMap_Supported_Version --- PASS: Test_Build_Scheduler_PolicyConfigMap_Supported_Version (0.00s) PASS ok k8s.io/kops/pkg/model/components 0.053s ? k8s.io/kops/pkg/model/components/addonmanifests [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awscloudcontrollermanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsebscsidriver [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsloadbalancercontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/clusterautoscaler [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/dnscontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/externaldns [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/karpenter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/kuberouter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/nodeterminationhandler [no test files] === RUN Test_RunEtcdManagerBuilder I0803 22:17:19.729982 19759 featureflag.go:162] FeatureFlag "ImageDigest"=false === RUN Test_RunEtcdManagerBuilder/basedir=tests/minimal === RUN Test_RunEtcdManagerBuilder/basedir=tests/pollinterval === RUN Test_RunEtcdManagerBuilder/basedir=tests/proxy === RUN Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings W0803 22:17:19.756890 19759 model.go:347] overriding log level in manifest (embedded etcd manifest), new level is 3 W0803 22:17:19.757028 19759 model.go:480] overloading ENV var in manifest (embedded etcd manifest) with ETCD_QUOTA_BACKEND_BYTES=10737418240 W0803 22:17:19.759822 19759 model.go:347] overriding log level in manifest (embedded etcd manifest), new level is 3 W0803 22:17:19.759991 19759 model.go:480] overloading ENV var in manifest (embedded etcd manifest) with ETCD_QUOTA_BACKEND_BYTES=10737418240 --- PASS: Test_RunEtcdManagerBuilder (0.03s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/minimal (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/pollinterval (0.00s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/proxy (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings (0.01s) PASS ok k8s.io/kops/pkg/model/components/etcdmanager 0.089s ? k8s.io/kops/pkg/model/components/kopscontroller [no test files] === RUN Test_RunKubeApiserverBuilder I0803 22:17:19.696176 19748 featureflag.go:162] FeatureFlag "ImageDigest"=false === RUN Test_RunKubeApiserverBuilder/basedir=tests/minimal --- PASS: Test_RunKubeApiserverBuilder (0.01s) --- PASS: Test_RunKubeApiserverBuilder/basedir=tests/minimal (0.01s) PASS ok k8s.io/kops/pkg/model/components/kubeapiserver 0.071s === RUN TestDefaultInstanceGroupVolumeSize --- PASS: TestDefaultInstanceGroupVolumeSize (0.00s) PASS ok k8s.io/kops/pkg/model/defaults 0.050s ? k8s.io/kops/pkg/model/domodel [no test files] ? k8s.io/kops/pkg/model/gcemodel [no test files] ? k8s.io/kops/pkg/model/hetznermodel [no test files] === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) === RUN TestPolicyGeneration --- PASS: TestPolicyGeneration (0.00s) === RUN TestEmptyPolicy --- PASS: TestEmptyPolicy (0.00s) PASS ok k8s.io/kops/pkg/model/iam 0.113s === RUN TestServerGroupBuilder I0803 22:17:20.647108 19959 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:20.676754 19959 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:20.686807 19959 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:20.709186 19959 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:20.835564 19959 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:20.860873 19959 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:20.873634 19959 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:20.884842 19959 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:20.894121 19959 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:20.934634 19959 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:20.943476 19959 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:20.952964 19959 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:20.960084 19959 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:20.968548 19959 featureflag.go:162] FeatureFlag "ImageDigest"=false --- PASS: TestServerGroupBuilder (0.33s) PASS ok k8s.io/kops/pkg/model/openstackmodel 0.421s === RUN Test_NodeUpTabs --- PASS: Test_NodeUpTabs (0.00s) PASS ok k8s.io/kops/pkg/model/resources 0.036s ? k8s.io/kops/pkg/nodeidentity [no test files] ? k8s.io/kops/pkg/nodeidentity/aws [no test files] === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetVMSSNameFromProviderID === RUN TestGetVMSSNameFromProviderID/test_case_0 === RUN TestGetVMSSNameFromProviderID/test_case_1 === RUN TestGetVMSSNameFromProviderID/test_case_2 --- PASS: TestGetVMSSNameFromProviderID (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_0 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_1 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_2 (0.00s) === RUN TestIdentifyNode === RUN TestIdentifyNode/test_case_0 === RUN TestIdentifyNode/test_case_1 --- PASS: TestIdentifyNode (0.00s) --- PASS: TestIdentifyNode/test_case_0 (0.00s) --- PASS: TestIdentifyNode/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/nodeidentity/azure 0.008s ? k8s.io/kops/pkg/nodeidentity/do [no test files] ? k8s.io/kops/pkg/nodeidentity/gce [no test files] ? k8s.io/kops/pkg/nodeidentity/hetzner [no test files] ? k8s.io/kops/pkg/nodeidentity/openstack [no test files] === RUN TestBuildNodeLabels === RUN TestBuildNodeLabels/RoleControlPlane === RUN TestBuildNodeLabels/RoleNode --- PASS: TestBuildNodeLabels (0.00s) --- PASS: TestBuildNodeLabels/RoleControlPlane (0.00s) --- PASS: TestBuildNodeLabels/RoleNode (0.00s) PASS ok k8s.io/kops/pkg/nodelabels 0.044s === RUN TestGenerateCertificate === RUN TestGenerateCertificate/selfsigned === RUN TestGenerateCertificate/client === RUN TestGenerateCertificate/server --- PASS: TestGenerateCertificate (0.01s) --- PASS: TestGenerateCertificate/selfsigned (0.01s) --- PASS: TestGenerateCertificate/client (0.00s) --- PASS: TestGenerateCertificate/server (0.00s) === RUN TestCertificateRoundTrip --- PASS: TestCertificateRoundTrip (0.00s) === RUN TestIssueCert === RUN TestIssueCert/ca === RUN TestIssueCert/client === RUN TestIssueCert/clientOneYear === RUN TestIssueCert/#00 === RUN TestIssueCert/server --- PASS: TestIssueCert (3.03s) --- PASS: TestIssueCert/ca (0.32s) --- PASS: TestIssueCert/client (0.58s) --- PASS: TestIssueCert/clientOneYear (2.13s) --- PASS: TestIssueCert/#00 (0.00s) --- PASS: TestIssueCert/server (0.00s) === RUN TestPrivateKeyRoundTripRSA === RUN TestPrivateKeyRoundTripRSA/rsa === RUN TestPrivateKeyRoundTripRSA/ecdsa --- PASS: TestPrivateKeyRoundTripRSA (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/rsa (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/ecdsa (0.00s) === RUN Test_AWSFingerprint_RsaKey1 --- PASS: Test_AWSFingerprint_RsaKey1 (0.00s) === RUN Test_AWSFingerprint_RsaKeyEncrypted --- PASS: Test_AWSFingerprint_RsaKeyEncrypted (0.00s) === RUN Test_AWSFingerprint_Ed25519Key --- PASS: Test_AWSFingerprint_Ed25519Key (0.00s) === RUN Test_AWSFingerprint_TrickyWhitespace --- PASS: Test_AWSFingerprint_TrickyWhitespace (0.00s) === RUN Test_AWSFingerprint_DsaKey --- PASS: Test_AWSFingerprint_DsaKey (0.00s) === RUN Test_OpenSSHFingerprint_RsaKey1 --- PASS: Test_OpenSSHFingerprint_RsaKey1 (0.00s) PASS ok k8s.io/kops/pkg/pki 3.088s === RUN TestLongDesc --- PASS: TestLongDesc (0.00s) PASS ok k8s.io/kops/pkg/pretty 0.053s ? k8s.io/kops/pkg/rbac [no test files] ? k8s.io/kops/pkg/resolver [no test files] ? k8s.io/kops/pkg/resources [no test files] === RUN TestAddUntaggedRouteTables I0803 22:17:18.568598 19549 routetable.go:53] DescribeRouteTables: { } W0803 22:17:18.568742 19549 tags.go:67] cluster tag not found on route-table:rtb-1234 I0803 22:17:18.568760 19549 aws.go:225] Skipping route table in VPC, but with wrong cluster tag ("other.example.com") --- PASS: TestAddUntaggedRouteTables (0.00s) === RUN TestListRouteTables I0803 22:17:18.568859 19549 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["me.example.com"] }] } I0803 22:17:18.568960 19549 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/me.example.com"] }] } --- PASS: TestListRouteTables (0.00s) === RUN TestSharedVolume I0803 22:17:18.569043 19549 volumes.go:33] CreateVolume: { TagSpecifications: [{ ResourceType: "volume", Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "shared" }] }] } I0803 22:17:18.569123 19549 volumes.go:33] CreateVolume: { TagSpecifications: [{ Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "owned" }] }] } I0803 22:17:18.569159 19549 volumes.go:128] DescribeVolumes: { } W0803 22:17:18.569186 19549 aws.go:2239] (new) cluster tag not found on volume:vol-2 aws_test.go:192: 2 --- PASS: TestSharedVolume (0.00s) === RUN TestMatchesElbTags --- PASS: TestMatchesElbTags (0.00s) === RUN TestListVPCs I0803 22:17:18.569304 19549 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:18.569501 19549 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:18.569926 19549 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:18.569982 19549 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:18.570028 19549 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:18.570064 19549 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:18.570080 19549 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:18.570111 19549 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:18.570133 19549 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:18.570153 19549 tags.go:42] CreateTags { Resources: ["vpc-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "legacy.example.com" }] } I0803 22:17:18.570182 19549 tags.go:42] CreateTags { Resources: ["vpc-shared"], Tags: [{ Key: "kubernetes.io/cluster/shared.example.com", Value: "shared" }] } I0803 22:17:18.570209 19549 tags.go:42] CreateTags { Resources: ["vpc-shared-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "shared-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/shared-with-legacy.example.com", Value: "shared" }] } I0803 22:17:18.570249 19549 tags.go:42] CreateTags { Resources: ["vpc-owned"], Tags: [{ Key: "kubernetes.io/cluster/owned.example.com", Value: "owned" }] } I0803 22:17:18.570270 19549 tags.go:42] CreateTags { Resources: ["vpc-owned-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "owned-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/owned-with-legacy.example.com", Value: "owned" }] } I0803 22:17:18.570314 19549 tags.go:42] CreateTags { Resources: ["vpc-other"], Tags: [{ Key: "KubernetesCluster", Value: "other.example.com" },{ Key: "kubernetes.io/cluster/other.example.com", Value: "shared" }] } I0803 22:17:18.570368 19549 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["mismatch.example.com"] }] } I0803 22:17:18.570392 19549 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/mismatch.example.com"] }] } I0803 22:17:18.570414 19549 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["legacy.example.com"] }] } I0803 22:17:18.570434 19549 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/legacy.example.com"] }] } I0803 22:17:18.570457 19549 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared-with-legacy.example.com"] }] } I0803 22:17:18.570477 19549 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared-with-legacy.example.com"] }] } I0803 22:17:18.570499 19549 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared.example.com"] }] } I0803 22:17:18.570517 19549 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared.example.com"] }] } I0803 22:17:18.570538 19549 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned-with-legacy.example.com"] }] } I0803 22:17:18.570565 19549 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned-with-legacy.example.com"] }] } I0803 22:17:18.570589 19549 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned.example.com"] }] } I0803 22:17:18.570613 19549 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned.example.com"] }] } --- PASS: TestListVPCs (0.00s) PASS ok k8s.io/kops/pkg/resources/aws 0.066s === RUN TestListResourcesAzure --- PASS: TestListResourcesAzure (0.00s) === RUN TestIsOwnedByCluster === RUN TestIsOwnedByCluster/test_case_0 === RUN TestIsOwnedByCluster/test_case_1 === RUN TestIsOwnedByCluster/test_case_2 === RUN TestIsOwnedByCluster/test_case_3 --- PASS: TestIsOwnedByCluster (0.00s) --- PASS: TestIsOwnedByCluster/test_case_0 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_1 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_2 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/resources/azure 0.028s ? k8s.io/kops/pkg/resources/digitalocean [no test files] === RUN TestNameMatch --- PASS: TestNameMatch (0.00s) === RUN TestMatchesClusterNameWithUUID --- PASS: TestMatchesClusterNameWithUUID (0.00s) PASS ok k8s.io/kops/pkg/resources/gce 0.047s ? k8s.io/kops/pkg/resources/hetzner [no test files] ? k8s.io/kops/pkg/resources/openstack [no test files] ? k8s.io/kops/pkg/resources/ops [no test files] ? k8s.io/kops/pkg/resources/spotinst [no test files] ? k8s.io/kops/pkg/sshcredentials [no test files] === RUN TestEscapeCommand --- PASS: TestEscapeCommand (0.00s) === RUN TestRawManifest --- PASS: TestRawManifest (0.00s) === RUN TestRawMixedManifest --- PASS: TestRawMixedManifest (0.00s) === RUN TestKeyPairOnlyManifest --- PASS: TestKeyPairOnlyManifest (0.00s) === RUN TestUnitFileExtensionValid === RUN TestUnitFileExtensionValid/valid_extension:_.automount === RUN TestUnitFileExtensionValid/valid_extension:_.device === RUN TestUnitFileExtensionValid/valid_extension:_.mount === RUN TestUnitFileExtensionValid/valid_extension:_.path === RUN TestUnitFileExtensionValid/valid_extension:_.scope === RUN TestUnitFileExtensionValid/valid_extension:_.service === RUN TestUnitFileExtensionValid/valid_extension:_.slice === RUN TestUnitFileExtensionValid/valid_extension:_.socket === RUN TestUnitFileExtensionValid/valid_extension:_.swap === RUN TestUnitFileExtensionValid/valid_extension:_.target === RUN TestUnitFileExtensionValid/valid_extension:_.timer === RUN TestUnitFileExtensionValid/invalid_extension:_(no_extension) === RUN TestUnitFileExtensionValid/invalid_extension:_.not-valid --- PASS: TestUnitFileExtensionValid (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.automount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.device (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.mount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.path (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.scope (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.service (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.slice (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.socket (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.swap (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.target (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.timer (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_(no_extension) (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_.not-valid (0.00s) PASS ok k8s.io/kops/pkg/systemd 0.049s ? k8s.io/kops/pkg/templates [no test files] ? k8s.io/kops/pkg/testutils [no test files] ? k8s.io/kops/pkg/testutils/golden [no test files] ? k8s.io/kops/pkg/tokens [no test files] === RUN TestTruncateString === RUN TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false --- PASS: TestTruncateString (0.00s) --- PASS: TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false (0.00s) PASS ok k8s.io/kops/pkg/truncate 0.036s ? k8s.io/kops/pkg/try [no test files] === RUN TestJoin --- PASS: TestJoin (0.00s) PASS ok k8s.io/kops/pkg/urls 0.120s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) PASS ok k8s.io/kops/pkg/util/stringorslice 0.111s === RUN Test_Overlap --- PASS: Test_Overlap (0.00s) === RUN Test_BelongsTo --- PASS: Test_BelongsTo (0.00s) === RUN Test_SplitInto8 --- PASS: Test_SplitInto8 (0.00s) PASS ok k8s.io/kops/pkg/util/subnet 0.074s === RUN TestRenderGeneralOK --- PASS: TestRenderGeneralOK (0.01s) === RUN TestRenderMissingValue --- PASS: TestRenderMissingValue (0.00s) === RUN TestRenderIndent --- PASS: TestRenderIndent (0.00s) === RUN TestRenderChannelFunctions --- PASS: TestRenderChannelFunctions (0.00s) === RUN TestRenderSnippet --- PASS: TestRenderSnippet (0.00s) === RUN TestRenderContext --- PASS: TestRenderContext (0.00s) === RUN TestAllowForMissingVars --- PASS: TestAllowForMissingVars (0.00s) === RUN TestRenderIntegration --- PASS: TestRenderIntegration (0.00s) PASS ok k8s.io/kops/pkg/util/templater 0.041s === RUN Test_ValidateCloudGroupMissing --- PASS: Test_ValidateCloudGroupMissing (0.00s) === RUN Test_ValidateNodesNotEnough --- PASS: Test_ValidateNodesNotEnough (0.00s) === RUN Test_ValidateDetachedNodesDontCount --- PASS: Test_ValidateDetachedNodesDontCount (0.00s) === RUN Test_ValidateNodeNotReady --- PASS: Test_ValidateNodeNotReady (0.00s) === RUN Test_ValidateMastersNotEnough --- PASS: Test_ValidateMastersNotEnough (0.00s) === RUN Test_ValidateMasterNotReady --- PASS: Test_ValidateMasterNotReady (0.00s) === RUN Test_ValidateMasterStaticPods --- PASS: Test_ValidateMasterStaticPods (0.00s) === RUN Test_ValidateNoPodFailures --- PASS: Test_ValidateNoPodFailures (0.00s) === RUN Test_ValidatePodFailure === RUN Test_ValidatePodFailure/pending-node-kube-system === RUN Test_ValidatePodFailure/pending-node-kube-system#01 === RUN Test_ValidatePodFailure/pending-node-kube-system#02 === RUN Test_ValidatePodFailure/pending-node-otherNamespace === RUN Test_ValidatePodFailure/pending-node-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-node-otherNamespace#02 === RUN Test_ValidatePodFailure/pending-cluster-kube-system === RUN Test_ValidatePodFailure/pending-cluster-kube-system#01 === RUN Test_ValidatePodFailure/pending-cluster-kube-system#02 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-node-kube-system === RUN Test_ValidatePodFailure/notready-node-kube-system#01 === RUN Test_ValidatePodFailure/notready-node-kube-system#02 === RUN Test_ValidatePodFailure/notready-node-otherNamespace === RUN Test_ValidatePodFailure/notready-node-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-node-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-cluster-kube-system === RUN Test_ValidatePodFailure/notready-cluster-kube-system#01 === RUN Test_ValidatePodFailure/notready-cluster-kube-system#02 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-node-kube-system === RUN Test_ValidatePodFailure/unknown-node-kube-system#01 === RUN Test_ValidatePodFailure/unknown-node-kube-system#02 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#01 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#02 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 --- PASS: Test_ValidatePodFailure (0.01s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 (0.00s) === RUN Test_ValidateBastionNodes === RUN Test_ValidateBastionNodes/instancegroup's_nodes_not_ready === RUN Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready --- PASS: Test_ValidateBastionNodes (0.00s) --- PASS: Test_ValidateBastionNodes/instancegroup's_nodes_not_ready (0.00s) --- PASS: Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready (0.00s) === RUN Test_ValidateDetachedNodesNotValidated --- PASS: Test_ValidateDetachedNodesNotValidated (0.00s) PASS ok k8s.io/kops/pkg/validation 0.068s ? k8s.io/kops/pkg/values [no test files] ? k8s.io/kops/pkg/wellknownoperators [no test files] ? k8s.io/kops/pkg/wellknownports [no test files] ? k8s.io/kops/pkg/wellknownusers [no test files] ? k8s.io/kops/pkg/zones [no test files] ? k8s.io/kops/protokube/cmd/protokube [no test files] ? k8s.io/kops/protokube/pkg/etcd [no test files] ? k8s.io/kops/protokube/pkg/gossip [no test files] ? k8s.io/kops/protokube/pkg/gossip/aws [no test files] === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetInternalIP --- PASS: TestGetInternalIP (0.00s) === RUN TestGetSeeds --- PASS: TestGetSeeds (0.00s) PASS ok k8s.io/kops/protokube/pkg/gossip/azure 0.004s ? k8s.io/kops/protokube/pkg/gossip/dns [no test files] === RUN TestRemovesDuplicateGuardedBlocks W0803 22:16:39.690631 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.690700 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.690713 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.690724 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.690735 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.690745 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.690755 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.690766 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.690776 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.690787 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.690798 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.690808 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.690818 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.690828 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.690838 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.690849 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.693343 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.693377 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.693453 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.693470 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.693538 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.693555 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.693618 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.693633 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.693696 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.693712 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.693786 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.693802 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.693870 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.693888 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.693977 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.693995 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.694059 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.694076 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.694141 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.694158 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.694221 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.694236 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.694323 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.694339 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.694402 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.694419 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.694481 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.694496 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.694561 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.694577 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.694643 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.694659 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.694723 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.694742 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.694803 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.694818 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.694881 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.694908 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.695011 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.695028 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.695091 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.695106 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.695169 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.695187 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.695251 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.695267 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.695331 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.695346 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.695408 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.695424 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.695563 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.695599 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.695667 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.695683 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.695745 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.695759 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.695818 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.695837 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.695899 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.695914 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.695972 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.695990 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.696060 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.696078 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.696144 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.696160 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.696224 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.696240 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.696296 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.696308 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.696358 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.696371 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.696449 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.696464 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.696533 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.696547 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.696607 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.696625 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.696691 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.696708 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.696801 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.696819 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.696905 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.696922 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.696987 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.697003 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.697066 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.697082 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.697145 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.697162 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.697222 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.697239 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.697312 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.697329 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.697392 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.697406 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.697469 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.697482 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.697543 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.697561 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.697622 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.697637 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.697716 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.697732 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.697793 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.697809 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.697875 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.698496 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.698586 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.698603 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.698669 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.698686 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.698747 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.698764 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.698835 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.698854 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.698913 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.698928 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.698985 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.699001 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.699061 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.699076 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.699132 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.699147 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.699231 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.699249 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.699317 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.699334 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.699398 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.699415 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.699476 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.699492 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.699552 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.699569 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.699630 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.699646 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.699728 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.699744 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.699803 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.699821 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.699881 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.699897 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.699959 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.699975 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.700039 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.700055 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.700118 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.700154 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.700219 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.700235 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.700298 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.700314 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.700375 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.700391 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.700456 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.700472 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.700534 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.700550 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.700613 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.700628 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.700688 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.700705 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.700768 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.700783 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.700843 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.700860 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.700951 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.700969 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.701030 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.701045 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.701104 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.701120 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.701180 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.701197 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.701256 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.701272 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.701332 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.701349 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.701411 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.701428 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.701484 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.701500 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.701556 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.701571 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.701636 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.701653 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.701727 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.701744 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.701806 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.701822 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.701890 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.701906 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.701971 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.701986 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.702047 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.702063 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.702164 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.702183 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRemovesDuplicateGuardedBlocks (0.01s) === RUN TestRecoversFromBadNesting W0803 22:16:39.702925 4587 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0803 22:16:39.702939 4587 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0803 22:16:39.702949 4587 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0803 22:16:39.702959 4587 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0803 22:16:39.702968 4587 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0803 22:16:39.702979 4587 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0803 22:16:39.702988 4587 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0803 22:16:39.703018 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.703031 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.703042 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.703051 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.703061 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.703070 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.703080 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.703092 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.703102 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.703112 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.703124 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.703136 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.703147 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.703159 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.703169 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.703179 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.711630 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.711660 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.711734 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.711751 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.711816 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.711834 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.711917 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.711934 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.712023 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.712040 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.712104 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.712130 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.712193 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.712211 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.712280 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.712296 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.712359 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.712376 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.712438 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.712454 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.712521 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.712537 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.712601 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.712620 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.712702 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.712720 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.712786 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.712803 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.712868 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.712883 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.712946 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.712963 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.713026 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.713043 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.713106 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.713123 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.713188 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.713203 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.713269 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.713284 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.713345 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.713362 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.713426 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.713443 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.713517 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.713534 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.713599 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.713615 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.713680 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.713694 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.713758 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.713773 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.713834 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.713850 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.713935 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.713951 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.714046 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.714065 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.714133 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.714149 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.714212 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.714228 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.714292 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.714308 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.714370 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.714386 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.714449 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.714464 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.714528 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.714543 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.714611 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.714627 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.714692 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.714708 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.714771 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.714787 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.714852 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.714867 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.714942 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.714958 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.715022 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.715037 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.715100 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.715118 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.715199 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.715216 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.715284 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.715299 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.715362 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.715377 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.715453 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.715475 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.715535 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.715550 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.715606 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.715620 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.715674 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.715687 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.715739 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.715753 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.715820 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.715836 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.715895 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.715909 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.715962 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.715976 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.716059 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.716073 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.716128 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.716141 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.716195 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.716209 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.716263 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.716277 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.716332 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.716345 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.716400 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.716414 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.716470 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.716484 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.716538 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.716559 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.716615 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.716629 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.716682 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.716695 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.716754 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.716769 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.716823 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.716837 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.716892 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.716906 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.716962 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.716979 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.717036 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.717049 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.717102 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.717117 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.717171 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.717184 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.717236 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.717250 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.717317 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.717333 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.717385 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.717398 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.717451 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.717465 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.717521 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.717534 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.717590 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.717603 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.717660 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.717674 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.717753 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.717769 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.717833 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.717850 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.717912 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.717932 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.717993 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.718011 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.718101 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.718118 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.718185 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.718200 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.718267 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.718283 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.718347 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.718362 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.718424 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.718439 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.718502 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.718517 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.718578 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.718593 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.718655 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.718671 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.718737 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.718755 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.718827 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.718847 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.718914 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.718928 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.718991 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.719005 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.719064 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.719079 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.719143 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.719157 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.719219 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.719233 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.719294 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.719308 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.719378 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.719393 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0803 22:16:39.719456 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0803 22:16:39.719469 4587 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRecoversFromBadNesting (0.02s) PASS ok k8s.io/kops/protokube/pkg/gossip/dns/hosts 0.076s ? k8s.io/kops/protokube/pkg/gossip/dns/provider [no test files] ? k8s.io/kops/protokube/pkg/gossip/do [no test files] ? k8s.io/kops/protokube/pkg/gossip/hetzner [no test files] ? k8s.io/kops/protokube/pkg/gossip/memberlist [no test files] ? k8s.io/kops/protokube/pkg/gossip/mesh [no test files] ? k8s.io/kops/protokube/pkg/gossip/openstack [no test files] ? k8s.io/kops/protokube/pkg/hostmount [no test files] ? k8s.io/kops/protokube/pkg/protokube [no test files] === RUN TestSerializeEmptyCluster --- PASS: TestSerializeEmptyCluster (0.00s) PASS ok k8s.io/kops/tests/codecs 0.034s ? k8s.io/kops/tests/fuzz [no test files] === RUN TestKopsUpgrades --- PASS: TestKopsUpgrades (0.01s) === RUN TestKubernetesUpgrades --- PASS: TestKubernetesUpgrades (0.00s) === RUN TestFindImage --- PASS: TestFindImage (0.00s) === RUN TestRecommendedKubernetesVersion --- PASS: TestRecommendedKubernetesVersion (0.00s) === RUN TestOrdering --- PASS: TestOrdering (0.00s) === RUN TestChannelsSelfConsistent --- PASS: TestChannelsSelfConsistent (0.02s) === RUN TestChannelImages === RUN TestChannelImages/stable-channel === RUN TestChannelImages/alpha-channel --- PASS: TestChannelImages (0.01s) --- PASS: TestChannelImages/stable-channel (0.00s) --- PASS: TestChannelImages/alpha-channel (0.00s) PASS ok k8s.io/kops/tests/integration/channel 0.061s ? k8s.io/kops/tests/integration/channel/simple [no test files] === RUN TestConversionMinimal === RUN TestConversionMinimal/legacy-v1alpha2-v1alpha2 --- PASS: TestConversionMinimal (0.01s) --- PASS: TestConversionMinimal/legacy-v1alpha2-v1alpha2 (0.01s) PASS ok k8s.io/kops/tests/integration/conversion 0.036s ? k8s.io/kops/upup/models [no test files] === RUN Test_tryResourceAsString --- PASS: Test_tryResourceAsString (0.00s) === RUN Test_DryrunTarget_PrintReport --- PASS: Test_DryrunTarget_PrintReport (0.00s) === RUN TestWriteFile I0803 22:17:00.566124 14562 files.go:57] Writing file "/tmp/TestWriteFile2926287940/001/SubDir/test1.tmp" I0803 22:17:00.566263 14562 files.go:113] Changing file mode for "/tmp/TestWriteFile2926287940/001/SubDir/test1.tmp" to -rw-r--r-- --- PASS: TestWriteFile (0.00s) === RUN TestBigInt_Format --- PASS: TestBigInt_Format (0.00s) === RUN TestVFSCAStoreRoundTrip --- PASS: TestVFSCAStoreRoundTrip (0.00s) === RUN TestVFSCAStoreRoundTripWithVault vfs_castore_test.go:161: No vault dev token set --- SKIP: TestVFSCAStoreRoundTripWithVault (0.00s) === RUN TestAddItem === RUN TestAddItem/first_primary === RUN TestAddItem/first_secondary === RUN TestAddItem/firstBigSerial_primary === RUN TestAddItem/firstBigSerial_secondary === RUN TestAddItem/after_primary === RUN TestAddItem/after_secondary === RUN TestAddItem/bigSerialAfter_primary === RUN TestAddItem/bigSerialAfter_secondary === RUN TestAddItem/before_primary === RUN TestAddItem/before_secondary === RUN TestAddItem/first_certonly_primary === RUN TestAddItem/first_certonly_secondary === RUN TestAddItem/after_certonly_primary === RUN TestAddItem/after_certonly_secondary === RUN TestAddItem/before_certonly_primary === RUN TestAddItem/before_certonly_secondary --- PASS: TestAddItem (0.00s) --- PASS: TestAddItem/first_primary (0.00s) --- PASS: TestAddItem/first_secondary (0.00s) --- PASS: TestAddItem/firstBigSerial_primary (0.00s) --- PASS: TestAddItem/firstBigSerial_secondary (0.00s) --- PASS: TestAddItem/after_primary (0.00s) --- PASS: TestAddItem/after_secondary (0.00s) --- PASS: TestAddItem/bigSerialAfter_primary (0.00s) --- PASS: TestAddItem/bigSerialAfter_secondary (0.00s) --- PASS: TestAddItem/before_primary (0.00s) --- PASS: TestAddItem/before_secondary (0.00s) --- PASS: TestAddItem/first_certonly_primary (0.00s) --- PASS: TestAddItem/first_certonly_secondary (0.00s) --- PASS: TestAddItem/after_certonly_primary (0.00s) --- PASS: TestAddItem/after_certonly_secondary (0.00s) --- PASS: TestAddItem/before_certonly_primary (0.00s) --- PASS: TestAddItem/before_certonly_secondary (0.00s) PASS ok k8s.io/kops/upup/pkg/fi 0.027s === RUN TestBootstrapChannelBuilder_BuildTasks I0803 22:17:26.039850 20346 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:26.040098 20346 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:26.040143 20346 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:26.040178 20346 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:26.040232 20346 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:26.040268 20346 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:26.040285 20346 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:26.040310 20346 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:26.040335 20346 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:26.047325 20346 zones.go:93] ListHostedZonesPages { } I0803 22:17:26.102979 20346 zones.go:93] ListHostedZonesPages { } I0803 22:17:26.236689 20346 zones.go:93] ListHostedZonesPages { } I0803 22:17:26.329808 20346 zones.go:93] ListHostedZonesPages { } I0803 22:17:26.431172 20346 zones.go:93] ListHostedZonesPages { } I0803 22:17:26.494570 20346 zones.go:93] ListHostedZonesPages { } I0803 22:17:26.567195 20346 zones.go:93] ListHostedZonesPages { } I0803 22:17:26.655906 20346 zones.go:93] ListHostedZonesPages { } I0803 22:17:26.771065 20346 zones.go:93] ListHostedZonesPages { } I0803 22:17:27.188212 20346 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_BuildTasks (1.20s) === RUN TestBootstrapChannelBuilder_ServiceAccountIAM I0803 22:17:27.241660 20346 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:27.241813 20346 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:27.241883 20346 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:27.241914 20346 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:27.241950 20346 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:27.241978 20346 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:27.241995 20346 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:27.242019 20346 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:27.242036 20346 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:27.242055 20346 featureflag.go:165] Unknown FeatureFlag "+UseServiceAccountExternalPermissions" I0803 22:17:27.243230 20346 zones.go:93] ListHostedZonesPages { } I0803 22:17:27.304157 20346 featureflag.go:165] Unknown FeatureFlag "-UseServiceAccountExternalPermissions" --- PASS: TestBootstrapChannelBuilder_ServiceAccountIAM (0.06s) === RUN TestBootstrapChannelBuilder_AWSCloudController I0803 22:17:27.304450 20346 featureflag.go:162] FeatureFlag "ImageDigest"=false I0803 22:17:27.304816 20346 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:27.304893 20346 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0803 22:17:27.304947 20346 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0803 22:17:27.304998 20346 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0803 22:17:27.305044 20346 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0803 22:17:27.305075 20346 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0803 22:17:27.305119 20346 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0803 22:17:27.305148 20346 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0803 22:17:27.307164 20346 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_AWSCloudController (0.10s) === RUN TestContainerdVersionUrlHash === RUN TestContainerdVersionUrlHash/1.3.4-amd64 === RUN TestContainerdVersionUrlHash/1.3.4-arm64 === RUN TestContainerdVersionUrlHash/1.3.10-amd64 === RUN TestContainerdVersionUrlHash/1.3.10-arm64 === RUN TestContainerdVersionUrlHash/1.4.1-amd64 === RUN TestContainerdVersionUrlHash/1.4.1-arm64 === RUN TestContainerdVersionUrlHash/1.4.4-amd64 === RUN TestContainerdVersionUrlHash/1.4.4-arm64 --- PASS: TestContainerdVersionUrlHash (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-arm64 (0.00s) === RUN TestContainerdVersionUrl === RUN TestContainerdVersionUrl/1.4.1- === RUN TestContainerdVersionUrl/1.4.1-arm === RUN TestContainerdVersionUrl/-amd64 === RUN TestContainerdVersionUrl/-arm64 === RUN TestContainerdVersionUrl/1.1.1-amd64 === RUN TestContainerdVersionUrl/1.1.1-arm64 === RUN TestContainerdVersionUrl/1.3.5-amd64 === RUN TestContainerdVersionUrl/1.3.5-arm64 === RUN TestContainerdVersionUrl/1.3.4-amd64 === RUN TestContainerdVersionUrl/1.3.4-arm64 === RUN TestContainerdVersionUrl/1.4.1-amd64 === RUN TestContainerdVersionUrl/1.4.1-arm64 === RUN TestContainerdVersionUrl/1.4.3-amd64 === RUN TestContainerdVersionUrl/1.4.3-arm64 --- PASS: TestContainerdVersionUrl (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1- (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionUrl/-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionHash === RUN TestContainerdVersionHash/1.4.1- === RUN TestContainerdVersionHash/1.4.1-arm === RUN TestContainerdVersionHash/-amd64 === RUN TestContainerdVersionHash/-arm64 === RUN TestContainerdVersionHash/1.1.1-amd64 === RUN TestContainerdVersionHash/1.1.1-arm64 === RUN TestContainerdVersionHash/1.3.5-amd64 === RUN TestContainerdVersionHash/1.3.5-arm64 === RUN TestContainerdVersionHash/1.4.1-amd64 === RUN TestContainerdVersionHash/1.4.1-arm64 === RUN TestContainerdVersionHash/1.4.3-amd64 === RUN TestContainerdVersionHash/1.4.3-arm64 --- PASS: TestContainerdVersionHash (0.00s) --- PASS: TestContainerdVersionHash/1.4.1- (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionHash/-amd64 (0.00s) --- PASS: TestContainerdVersionHash/-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionsHashesAmd64 containerd_test.go:316: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesAmd64 (0.00s) === RUN TestContainerdVersionsHashesArm64 containerd_test.go:331: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesArm64 (0.00s) === RUN TestDeepValidate_OK --- PASS: TestDeepValidate_OK (0.00s) === RUN TestDeepValidate_NoNodeZones --- PASS: TestDeepValidate_NoNodeZones (0.00s) === RUN TestDeepValidate_NoMasterZones --- PASS: TestDeepValidate_NoMasterZones (0.00s) === RUN TestDeepValidate_BadZone deepvalidate_test.go:57: Zone validation not checked by DeepValidate --- SKIP: TestDeepValidate_BadZone (0.00s) === RUN TestDeepValidate_MixedRegion deepvalidate_test.go:69: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_MixedRegion (0.00s) === RUN TestDeepValidate_RegionAsZone deepvalidate_test.go:83: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_RegionAsZone (0.00s) === RUN TestDeepValidate_NotIncludedZone --- PASS: TestDeepValidate_NotIncludedZone (0.00s) === RUN TestDeepValidate_DuplicateZones --- PASS: TestDeepValidate_DuplicateZones (0.00s) === RUN TestDeepValidate_ExtraMasterZone --- PASS: TestDeepValidate_ExtraMasterZone (0.00s) === RUN TestDeepValidate_EvenEtcdClusterSize --- PASS: TestDeepValidate_EvenEtcdClusterSize (0.00s) === RUN TestDeepValidate_MissingEtcdMember --- PASS: TestDeepValidate_MissingEtcdMember (0.00s) === RUN TestPopulateClusterSpec_Proxy --- PASS: TestPopulateClusterSpec_Proxy (0.00s) === RUN TestPrecreateDNSNames --- PASS: TestPrecreateDNSNames (0.00s) === RUN TestDockerVersionUrlHash === RUN TestDockerVersionUrlHash/19.03.13-amd64 === RUN TestDockerVersionUrlHash/19.03.13-arm64 === RUN TestDockerVersionUrlHash/18.06.3-amd64 === RUN TestDockerVersionUrlHash/18.06.3-arm64 === RUN TestDockerVersionUrlHash/17.03.1-amd64 === RUN TestDockerVersionUrlHash/17.03.1-arm64 --- PASS: TestDockerVersionUrlHash (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-arm64 (0.00s) === RUN TestDockerVersionUrl === RUN TestDockerVersionUrl/19.03.13- === RUN TestDockerVersionUrl/19.03.13-arm === RUN TestDockerVersionUrl/-amd64 === RUN TestDockerVersionUrl/-arm64 === RUN TestDockerVersionUrl/18.06.3-amd64 === RUN TestDockerVersionUrl/18.06.3-arm64 === RUN TestDockerVersionUrl/19.03.13-amd64 === RUN TestDockerVersionUrl/19.03.13-arm64 --- PASS: TestDockerVersionUrl (0.00s) --- PASS: TestDockerVersionUrl/19.03.13- (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm (0.00s) --- PASS: TestDockerVersionUrl/-amd64 (0.00s) --- PASS: TestDockerVersionUrl/-arm64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm64 (0.00s) === RUN TestDockerVersionHash === RUN TestDockerVersionHash/19.03.13- === RUN TestDockerVersionHash/19.03.13-arm === RUN TestDockerVersionHash/-amd64 === RUN TestDockerVersionHash/-arm64 === RUN TestDockerVersionHash/1.1.1-amd64 === RUN TestDockerVersionHash/1.1.1-arm64 === RUN TestDockerVersionHash/19.03.13-amd64 === RUN TestDockerVersionHash/19.03.13-arm64 --- PASS: TestDockerVersionHash (0.00s) --- PASS: TestDockerVersionHash/19.03.13- (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm (0.00s) --- PASS: TestDockerVersionHash/-amd64 (0.00s) --- PASS: TestDockerVersionHash/-arm64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm64 (0.00s) === RUN TestDockerVersionsHashesAmd64 docker_test.go:245: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesAmd64 (0.00s) === RUN TestDockerVersionsHashesArm64 docker_test.go:260: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesArm64 (0.00s) === RUN Test_FindCNIAssetFromEnvironmentVariable --- PASS: Test_FindCNIAssetFromEnvironmentVariable (0.00s) === RUN Test_FindCNIAssetFromDefaults118 --- PASS: Test_FindCNIAssetFromDefaults118 (0.18s) === RUN Test_FindCNIAssetFromDefaults122 --- PASS: Test_FindCNIAssetFromDefaults122 (0.12s) === RUN TestRemoveSharedPrefix --- PASS: TestRemoveSharedPrefix (0.00s) === RUN TestCreateEtcdCluster --- PASS: TestCreateEtcdCluster (0.00s) === RUN TestSetupNetworking W0803 22:17:27.727740 20346 new_cluster.go:980] flannel UDP mode is not recommended; consider flannel-vxlan instead --- PASS: TestSetupNetworking (0.00s) === RUN TestPopulateCluster_Default_NoError --- PASS: TestPopulateCluster_Default_NoError (0.00s) === RUN TestPopulateCluster_Subnets === RUN TestPopulateCluster_Subnets/100.64.0.0/10 === RUN TestPopulateCluster_Subnets/10.0.0.0/9 === RUN TestPopulateCluster_Subnets/10.0.0.0/8 === RUN TestPopulateCluster_Subnets/::/0 --- PASS: TestPopulateCluster_Subnets (0.01s) --- PASS: TestPopulateCluster_Subnets/100.64.0.0/10 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/9 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/8 (0.00s) --- PASS: TestPopulateCluster_Subnets/::/0 (0.00s) === RUN TestPopulateCluster_Docker_Spec --- PASS: TestPopulateCluster_Docker_Spec (0.00s) === RUN TestPopulateCluster_StorageDefault --- PASS: TestPopulateCluster_StorageDefault (0.00s) === RUN TestPopulateCluster_Custom_CIDR --- PASS: TestPopulateCluster_Custom_CIDR (0.00s) === RUN TestPopulateCluster_IsolateMasters --- PASS: TestPopulateCluster_IsolateMasters (0.00s) === RUN TestPopulateCluster_IsolateMastersFalse --- PASS: TestPopulateCluster_IsolateMastersFalse (0.00s) === RUN TestPopulateCluster_Name_Required --- PASS: TestPopulateCluster_Name_Required (0.00s) === RUN TestPopulateCluster_Zone_Required --- PASS: TestPopulateCluster_Zone_Required (0.00s) === RUN TestPopulateCluster_NetworkCIDR_Required --- PASS: TestPopulateCluster_NetworkCIDR_Required (0.00s) === RUN TestPopulateCluster_NonMasqueradeCIDR_Required --- PASS: TestPopulateCluster_NonMasqueradeCIDR_Required (0.00s) === RUN TestPopulateCluster_CloudProvider_Required --- PASS: TestPopulateCluster_CloudProvider_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidNil_Required --- PASS: TestPopulateCluster_TopologyInvalidNil_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidValue_Required --- PASS: TestPopulateCluster_TopologyInvalidValue_Required (0.00s) === RUN TestPopulateCluster_BastionInvalidMatchingValues_Required --- PASS: TestPopulateCluster_BastionInvalidMatchingValues_Required (0.00s) === RUN TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required --- PASS: TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required (0.00s) === RUN TestPopulateCluster_APIServerCount --- PASS: TestPopulateCluster_APIServerCount (0.00s) === RUN TestPopulateCluster_AnonymousAuth --- PASS: TestPopulateCluster_AnonymousAuth (0.00s) === RUN TestPopulateCluster_DockerVersion --- PASS: TestPopulateCluster_DockerVersion (0.00s) === RUN TestPopulateCluster_KubeController_High_Enough_Version --- PASS: TestPopulateCluster_KubeController_High_Enough_Version (0.00s) === RUN TestPopulateInstanceGroup_Name_Required --- PASS: TestPopulateInstanceGroup_Name_Required (0.00s) === RUN TestPopulateInstanceGroup_Role_Required --- PASS: TestPopulateInstanceGroup_Role_Required (0.00s) === RUN TestPopulateInstanceGroup_Image_Required I0803 22:17:27.763184 20346 populate_instancegroup_spec.go:323] Cannot set default Image for CloudProvider="aws" --- PASS: TestPopulateInstanceGroup_Image_Required (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision --- PASS: TestPopulateInstanceGroup_AddTaintsCollision (0.00s) === RUN TestPopulateInstanceGroup_AddTaints --- PASS: TestPopulateInstanceGroup_AddTaints (0.00s) === RUN TestMachineArchitecture === RUN TestMachineArchitecture/t2.micro-amd64 === RUN TestMachineArchitecture/t3.micro-amd64 === RUN TestMachineArchitecture/a1.large-arm64 --- PASS: TestMachineArchitecture (0.00s) --- PASS: TestMachineArchitecture/t2.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/t3.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/a1.large-arm64 (0.00s) === RUN TestRuncVersionsHashesAmd64 runc_test.go:28: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesAmd64 (0.00s) === RUN TestRuncVersionsHashesArm64 runc_test.go:43: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesArm64 (0.00s) === RUN Test_AssignSubnets I0803 22:17:27.763967 20346 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0803 22:17:27.763989 20346 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0803 22:17:27.764000 20346 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I0803 22:17:27.764019 20346 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I0803 22:17:27.764030 20346 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet a I0803 22:17:27.764050 20346 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I0803 22:17:27.764069 20346 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet b I0803 22:17:27.764088 20346 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0803 22:17:27.764100 20346 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0803 22:17:27.764119 20346 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0803 22:17:27.764131 20346 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0803 22:17:27.764150 20346 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0803 22:17:27.764175 20346 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0803 22:17:27.764195 20346 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0803 22:17:27.764206 20346 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I0803 22:17:27.764215 20346 subnets.go:185] Assigned CIDR 10.96.0.0/11 to subnet b I0803 22:17:27.764225 20346 subnets.go:185] Assigned CIDR 10.128.0.0/11 to subnet c I0803 22:17:27.764235 20346 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet c I0803 22:17:27.764246 20346 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0803 22:17:27.764257 20346 subnets.go:199] Assigned CIDR 10.4.0.0/14 to subnet b I0803 22:17:27.764268 20346 subnets.go:199] Assigned CIDR 10.8.0.0/14 to subnet c --- PASS: Test_AssignSubnets (0.00s) === RUN Test_TemplateFunctions_CloudControllerConfigArgv === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller (0.00s) === RUN Test_KarpenterInstanceTypes I0803 22:17:27.765334 20346 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: Test_KarpenterInstanceTypes (0.00s) === RUN Test_BuildMirroredAsset === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-amd64 === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-arm64 === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup --- PASS: Test_BuildMirroredAsset (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-amd64 (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-arm64 (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup (0.00s) === RUN TestValidateFull_Default_Validates --- PASS: TestValidateFull_Default_Validates (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_NoDot --- PASS: TestValidateFull_ClusterName_InvalidDNS_NoDot (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_Invalid --- PASS: TestValidateFull_ClusterName_InvalidDNS_Invalid (0.00s) === RUN TestValidateFull_ClusterName_Required --- PASS: TestValidateFull_ClusterName_Required (0.00s) === RUN TestValidateFull_UpdatePolicy_Valid === RUN TestValidateFull_UpdatePolicy_Valid/missing === RUN TestValidateFull_UpdatePolicy_Valid/automatic === RUN TestValidateFull_UpdatePolicy_Valid/external --- PASS: TestValidateFull_UpdatePolicy_Valid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/missing (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/automatic (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/external (0.00s) === RUN TestValidateFull_UpdatePolicy_Invalid === RUN TestValidateFull_UpdatePolicy_Invalid/empty === RUN TestValidateFull_UpdatePolicy_Invalid/populated --- PASS: TestValidateFull_UpdatePolicy_Invalid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/empty (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/populated (0.00s) === RUN Test_Validate_Kubenet_With_14 --- PASS: Test_Validate_Kubenet_With_14 (0.00s) === RUN TestValidate_ClusterName_Import --- PASS: TestValidate_ClusterName_Import (0.00s) === RUN TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity --- PASS: TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup 1.803s === RUN TestGetASGTagsToDelete --- PASS: TestGetASGTagsToDelete (0.00s) === RUN TestProcessCompare --- PASS: TestProcessCompare (0.00s) === RUN TestAutoscalingGroupTerraformRender I0803 22:17:13.615067 18560 target.go:112] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender1220518946/001 I0803 22:17:13.616537 18560 target.go:112] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender1220518946/001 --- PASS: TestAutoscalingGroupTerraformRender (0.00s) === RUN TestAutoscalingGroupCloudformationRender I0803 22:17:13.617548 18560 target.go:168] Cloudformation output is in /tmp/TestAutoscalingGroupCloudformationRender2715685452/001 --- PASS: TestAutoscalingGroupCloudformationRender (0.00s) === RUN TestGetEBSVolumeTagsToDelete --- PASS: TestGetEBSVolumeTagsToDelete (0.00s) === RUN TestSharedEgressOnlyInternetGatewayDoesNotRename I0803 22:17:13.618824 18560 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0803 22:17:13.618922 18560 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0803 22:17:13.618999 18560 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { TagSpecifications: [{ ResourceType: "egress-only-internet-gateway", Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] }], VpcId: "vpc-1" } I0803 22:17:13.619243 18560 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0803 22:17:13.619357 18560 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:17:13.619429 18560 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0803 22:17:13.619449 18560 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0803 22:17:13.619701 18560 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0803 22:17:13.619747 18560 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0803 22:17:13.619790 18560 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0803 22:17:13.619861 18560 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0803 22:17:13.619960 18560 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0803 22:17:13.620110 18560 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0803 22:17:13.620153 18560 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:17:13.620197 18560 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0803 22:17:13.620215 18560 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0803 22:17:13.620247 18560 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0803 22:17:13.620295 18560 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0803 22:17:13.620353 18560 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedEgressOnlyInternetGatewayDoesNotRename (0.00s) === RUN TestElasticIPCreate I0803 22:17:13.620661 18560 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0803 22:17:13.620707 18560 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0803 22:17:13.620820 18560 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0803 22:17:13.620867 18560 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0803 22:17:13.620894 18560 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:17:13.620914 18560 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0803 22:17:13.620981 18560 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0803 22:17:13.621162 18560 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0803 22:17:13.621211 18560 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0803 22:17:13.621236 18560 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:17:13.621266 18560 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0803 22:17:13.621326 18560 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0803 22:17:13.621500 18560 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "eip1" }] }] } I0803 22:17:13.621596 18560 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0803 22:17:13.621629 18560 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedElasticIpAllocationId", Value: "eipalloc-1" },{ Key: "AssociatedElasticIp", Value: "192.0.2.1" }] } I0803 22:17:13.621674 18560 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0803 22:17:13.621844 18560 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0803 22:17:13.621932 18560 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0803 22:17:13.622007 18560 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0803 22:17:13.622047 18560 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0803 22:17:13.622099 18560 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:17:13.622159 18560 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0803 22:17:13.622223 18560 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0803 22:17:13.622309 18560 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:17:13.622355 18560 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0803 22:17:13.622417 18560 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0803 22:17:13.622495 18560 address.go:120] DescribeAddresses: { Filters: [{ Name: "public-ip", Values: ["192.0.2.1"] }] } I0803 22:17:13.622545 18560 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0803 22:17:13.622603 18560 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestElasticIPCreate (0.00s) === RUN TestSharedInternetGatewayDoesNotRename I0803 22:17:13.622852 18560 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0803 22:17:13.622923 18560 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0803 22:17:13.622972 18560 internetgateways.go:66] CreateInternetGateway: { } I0803 22:17:13.623018 18560 tags.go:42] CreateTags { Resources: ["igw-1"], Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] } I0803 22:17:13.623187 18560 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0803 22:17:13.623259 18560 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:17:13.623327 18560 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0803 22:17:13.623366 18560 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0803 22:17:13.623480 18560 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0803 22:17:13.623532 18560 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0803 22:17:13.623597 18560 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0803 22:17:13.623700 18560 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0803 22:17:13.623798 18560 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0803 22:17:13.623966 18560 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0803 22:17:13.624034 18560 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:17:13.624099 18560 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0803 22:17:13.624137 18560 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0803 22:17:13.624189 18560 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0803 22:17:13.624249 18560 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0803 22:17:13.624348 18560 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedInternetGatewayDoesNotRename (0.00s) === RUN TestLaunchTemplateCloudformationRender I0803 22:17:13.625384 18560 target.go:168] Cloudformation output is in /tmp/TestLaunchTemplateCloudformationRender3411031256/001 I0803 22:17:13.625700 18560 target.go:168] Cloudformation output is in /tmp/TestLaunchTemplateCloudformationRender3411031256/001 --- PASS: TestLaunchTemplateCloudformationRender (0.00s) === RUN TestLaunchTemplateTerraformRender I0803 22:17:13.628331 18560 target.go:112] Terraform output is in /tmp/TestLaunchTemplateTerraformRender3814317673/001 I0803 22:17:13.636015 18560 target.go:112] Terraform output is in /tmp/TestLaunchTemplateTerraformRender3814317673/001 --- PASS: TestLaunchTemplateTerraformRender (0.01s) === RUN TestParseRemovalRule --- PASS: TestParseRemovalRule (0.00s) === RUN TestPortRemovalRule --- PASS: TestPortRemovalRule (0.00s) === RUN TestPortRemovalRule_Zero --- PASS: TestPortRemovalRule_Zero (0.00s) === RUN TestSecurityGroupCreate I0803 22:17:13.636808 18560 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0803 22:17:13.636885 18560 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0803 22:17:13.637019 18560 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0803 22:17:13.637070 18560 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0803 22:17:13.637098 18560 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:17:13.637126 18560 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0803 22:17:13.637203 18560 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0803 22:17:13.637413 18560 securitygroups.go:40] CreateSecurityGroup: { Description: "Description", GroupName: "sg1", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "sg1" }] }], VpcId: "vpc-1" } I0803 22:17:13.637464 18560 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0803 22:17:13.637503 18560 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0803 22:17:13.637659 18560 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0803 22:17:13.637706 18560 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0803 22:17:13.637756 18560 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0803 22:17:13.637775 18560 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0803 22:17:13.637802 18560 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:17:13.637829 18560 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0803 22:17:13.637873 18560 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0803 22:17:13.637951 18560 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSecurityGroupCreate (0.00s) === RUN Test_Subnet_ValidateRequired --- PASS: Test_Subnet_ValidateRequired (0.00s) === RUN Test_Subnet_CannotChangeSubnet --- PASS: Test_Subnet_CannotChangeSubnet (0.00s) === RUN TestSubnetCreate I0803 22:17:13.638363 18560 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0803 22:17:13.638411 18560 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0803 22:17:13.638509 18560 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0803 22:17:13.638574 18560 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0803 22:17:13.638613 18560 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:17:13.638640 18560 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0803 22:17:13.638685 18560 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0803 22:17:13.638820 18560 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0803 22:17:13.638872 18560 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0803 22:17:13.638903 18560 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:17:13.638930 18560 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0803 22:17:13.639112 18560 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0803 22:17:13.639160 18560 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0803 22:17:13.639211 18560 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0803 22:17:13.639229 18560 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0803 22:17:13.639256 18560 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:17:13.639283 18560 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0803 22:17:13.639335 18560 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0803 22:17:13.639397 18560 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:17:13.639423 18560 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSubnetCreate (0.00s) === RUN TestSubnetCreateIPv6 I0803 22:17:13.639726 18560 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0803 22:17:13.639776 18560 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0803 22:17:13.639876 18560 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0803 22:17:13.639921 18560 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0803 22:17:13.639950 18560 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:17:13.639976 18560 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0803 22:17:13.640025 18560 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0803 22:17:13.640220 18560 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0803 22:17:13.640179 18560 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:17:13.640373 18560 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0803 22:17:13.640405 18560 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:17:13.640432 18560 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0803 22:17:13.640494 18560 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0803 22:17:13.640765 18560 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0803 22:17:13.640821 18560 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0803 22:17:13.640882 18560 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0803 22:17:13.640904 18560 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0803 22:17:13.640938 18560 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:17:13.640972 18560 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0803 22:17:13.641028 18560 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:17:13.641088 18560 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0803 22:17:13.641162 18560 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:17:13.641198 18560 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6 (0.00s) === RUN TestSubnetCreateIPv6NetNum I0803 22:17:13.641538 18560 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0803 22:17:13.641585 18560 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0803 22:17:13.641708 18560 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0803 22:17:13.641763 18560 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0803 22:17:13.641793 18560 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:17:13.641820 18560 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0803 22:17:13.641866 18560 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0803 22:17:13.642077 18560 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0803 22:17:13.642149 18560 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0803 22:17:13.642182 18560 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:17:13.642238 18560 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:17:13.642325 18560 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0803 22:17:13.642358 18560 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0803 22:17:13.642545 18560 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0803 22:17:13.642595 18560 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0803 22:17:13.642648 18560 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0803 22:17:13.642668 18560 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0803 22:17:13.642702 18560 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:17:13.642728 18560 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0803 22:17:13.642769 18560 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:17:13.642845 18560 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0803 22:17:13.642952 18560 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:17:13.642982 18560 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6NetNum (0.00s) === RUN TestSharedSubnetCreateDoesNotCreateNew I0803 22:17:13.643141 18560 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "ExistingVPC" }] }] } I0803 22:17:13.643231 18560 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "ExistingSubnet" }] }], VpcId: "vpc-1" } I0803 22:17:13.643401 18560 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0803 22:17:13.643448 18560 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:17:13.643502 18560 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0803 22:17:13.643522 18560 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0803 22:17:13.643617 18560 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0803 22:17:13.643654 18560 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0803 22:17:13.643693 18560 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0803 22:17:13.643763 18560 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:17:13.643889 18560 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0803 22:17:13.643922 18560 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0803 22:17:13.643963 18560 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0803 22:17:13.644135 18560 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0803 22:17:13.644188 18560 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:17:13.644236 18560 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0803 22:17:13.644256 18560 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0803 22:17:13.644291 18560 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0803 22:17:13.644331 18560 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0803 22:17:13.644399 18560 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedSubnetCreateDoesNotCreateNew (0.00s) === RUN TestVPCCreate I0803 22:17:13.644674 18560 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0803 22:17:13.644721 18560 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0803 22:17:13.644827 18560 vpcs.go:89] CreateVpc: { CidrBlock: "172.21.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0803 22:17:13.644876 18560 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0803 22:17:13.644906 18560 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:17:13.644929 18560 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0803 22:17:13.645064 18560 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0803 22:17:13.645109 18560 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0803 22:17:13.645160 18560 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0803 22:17:13.645180 18560 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0803 22:17:13.645211 18560 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0803 22:17:13.645234 18560 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestVPCCreate (0.00s) === RUN Test4758 --- PASS: Test4758 (0.00s) === RUN TestSharedVPCAdditionalCIDR I0803 22:17:13.645457 18560 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "172.22.0.0/16", VpcId: "vpc-1" } I0803 22:17:13.645618 18560 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0803 22:17:13.645667 18560 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc-1"] }] } I0803 22:17:13.645720 18560 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0803 22:17:13.645739 18560 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0803 22:17:13.645771 18560 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestSharedVPCAdditionalCIDR (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awstasks 0.125s === RUN TestValidateRegion --- PASS: TestValidateRegion (0.00s) === RUN TestFindRegion --- PASS: TestFindRegion (0.00s) === RUN TestEC2TagSpecification === RUN TestEC2TagSpecification/No_tags === RUN TestEC2TagSpecification/simple_tag --- PASS: TestEC2TagSpecification (0.00s) --- PASS: TestEC2TagSpecification/No_tags (0.00s) --- PASS: TestEC2TagSpecification/simple_tag (0.00s) === RUN Test_GetResourceName32 --- PASS: Test_GetResourceName32 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awsup 0.042s === RUN TestZoneToLocation === RUN TestZoneToLocation/test_case_0 === RUN TestZoneToLocation/test_case_1 === RUN TestZoneToLocation/test_case_2 --- PASS: TestZoneToLocation (0.00s) --- PASS: TestZoneToLocation/test_case_0 (0.00s) --- PASS: TestZoneToLocation/test_case_1 (0.00s) --- PASS: TestZoneToLocation/test_case_2 (0.00s) === RUN TestZoneToAvailabilityZoneNumber === RUN TestZoneToAvailabilityZoneNumber/test_case_0 === RUN TestZoneToAvailabilityZoneNumber/test_case_1 === RUN TestZoneToAvailabilityZoneNumber/test_case_2 --- PASS: TestZoneToAvailabilityZoneNumber (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_0 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_1 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_2 (0.00s) === RUN TestFindEtcdStatus --- PASS: TestFindEtcdStatus (0.00s) === RUN TestGetCloudGroups --- PASS: TestGetCloudGroups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azure 0.029s === RUN TestDiskRenderAzure I0803 22:17:04.108972 15803 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRenderAzure (0.00s) === RUN TestDiskFind --- PASS: TestDiskFind (0.00s) === RUN TestDiskRun I0803 22:17:04.109289 15803 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRun (0.00s) === RUN TestDiskCheckChanges === RUN TestDiskCheckChanges/test_case_0 === RUN TestDiskCheckChanges/test_case_1 === RUN TestDiskCheckChanges/test_case_2 === RUN TestDiskCheckChanges/test_case_3 --- PASS: TestDiskCheckChanges (0.00s) --- PASS: TestDiskCheckChanges/test_case_0 (0.00s) --- PASS: TestDiskCheckChanges/test_case_1 (0.00s) --- PASS: TestDiskCheckChanges/test_case_2 (0.00s) --- PASS: TestDiskCheckChanges/test_case_3 (0.00s) === RUN TestLoadBalancerRenderAzure I0803 22:17:04.109494 15803 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRenderAzure (0.00s) === RUN TestLoadBalancerFind loadbalancer_test.go:122: *azuretasks.LoadBalancer {"Name":"loadbalancer","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Subnet":{"Name":"subnet","Lifecycle":"","ResourceGroup":null,"VirtualNetwork":null,"CIDR":null,"Shared":null},"External":true,"Tags":null,"ForAPIServer":false} --- PASS: TestLoadBalancerFind (0.00s) === RUN TestLoadBalancerRun I0803 22:17:04.109893 15803 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRun (0.00s) === RUN TestLoadBalancerCheckChanges === RUN TestLoadBalancerCheckChanges/test_case_0 === RUN TestLoadBalancerCheckChanges/test_case_1 === RUN TestLoadBalancerCheckChanges/test_case_2 === RUN TestLoadBalancerCheckChanges/test_case_3 --- PASS: TestLoadBalancerCheckChanges (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_0 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_1 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_2 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_3 (0.00s) === RUN TestPublicIPAddressRenderAzure I0803 22:17:04.110126 15803 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRenderAzure (0.00s) === RUN TestPublicIPAddressFind publicipaddress_test.go:100: *azuretasks.PublicIPAddress {"Name":"publicIPAddress","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Tags":null} --- PASS: TestPublicIPAddressFind (0.00s) === RUN TestPublicIPAddressRun I0803 22:17:04.110368 15803 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRun (0.00s) === RUN TestPublicIPAddressCheckChanges === RUN TestPublicIPAddressCheckChanges/test_case_0 === RUN TestPublicIPAddressCheckChanges/test_case_1 === RUN TestPublicIPAddressCheckChanges/test_case_2 === RUN TestPublicIPAddressCheckChanges/test_case_3 --- PASS: TestPublicIPAddressCheckChanges (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_0 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_1 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_2 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_3 (0.00s) === RUN TestResourceGroupRenderAzure I0803 22:17:04.110597 15803 resourcegroup.go:103] Creating a new Resource Group with name: rg I0803 22:17:04.110607 15803 resourcegroup.go:105] Updating a Resource Group with name: rg --- PASS: TestResourceGroupRenderAzure (0.00s) === RUN TestResourceGroupFind --- PASS: TestResourceGroupFind (0.00s) === RUN TestResourceGroupRun I0803 22:17:04.110781 15803 resourcegroup.go:103] Creating a new Resource Group with name: rg --- PASS: TestResourceGroupRun (0.00s) === RUN TestResourceGroupCheckChanges === RUN TestResourceGroupCheckChanges/test_case_0 === RUN TestResourceGroupCheckChanges/test_case_1 === RUN TestResourceGroupCheckChanges/test_case_2 === RUN TestResourceGroupCheckChanges/test_case_3 --- PASS: TestResourceGroupCheckChanges (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_0 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_1 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_2 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_3 (0.00s) === RUN TestRoleAssignmentRenderAzure --- PASS: TestRoleAssignmentRenderAzure (0.00s) === RUN TestRoleAssignmentFind --- PASS: TestRoleAssignmentFind (0.00s) === RUN TestRoleAssignmentFind_NoPrincipalID --- PASS: TestRoleAssignmentFind_NoPrincipalID (0.00s) === RUN TestRoleAssignmentCheckChanges === RUN TestRoleAssignmentCheckChanges/test_case_0 === RUN TestRoleAssignmentCheckChanges/test_case_1 === RUN TestRoleAssignmentCheckChanges/test_case_2 === RUN TestRoleAssignmentCheckChanges/test_case_3 --- PASS: TestRoleAssignmentCheckChanges (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_0 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_1 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_2 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_3 (0.00s) === RUN TestSubnetRenderAzure I0803 22:17:04.111301 15803 subnet.go:106] Creating a new Subnet with name: vnet --- PASS: TestSubnetRenderAzure (0.00s) === RUN TestSubnetFind --- PASS: TestSubnetFind (0.00s) === RUN TestSubnetCheckChanges === RUN TestSubnetCheckChanges/test_case_0 === RUN TestSubnetCheckChanges/test_case_1 === RUN TestSubnetCheckChanges/test_case_2 === RUN TestSubnetCheckChanges/test_case_3 --- PASS: TestSubnetCheckChanges (0.00s) --- PASS: TestSubnetCheckChanges/test_case_0 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_1 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_2 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_3 (0.00s) === RUN TestVirtualNetworkRenderAzure I0803 22:17:04.111561 15803 virtualnetwork.go:116] Creating a new Virtual Network with name: vnet --- PASS: TestVirtualNetworkRenderAzure (0.00s) === RUN TestVirtualNetworkFind --- PASS: TestVirtualNetworkFind (0.00s) === RUN TestVirtualNetworkRun I0803 22:17:04.111776 15803 virtualnetwork.go:116] Creating a new Virtual Network with name: rg --- PASS: TestVirtualNetworkRun (0.00s) === RUN TestVirtualNetworkCheckChanges === RUN TestVirtualNetworkCheckChanges/test_case_0 === RUN TestVirtualNetworkCheckChanges/test_case_1 === RUN TestVirtualNetworkCheckChanges/test_case_2 === RUN TestVirtualNetworkCheckChanges/test_case_3 --- PASS: TestVirtualNetworkCheckChanges (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_0 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_1 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_2 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_3 (0.00s) === RUN TestSubnetIDParse --- PASS: TestSubnetIDParse (0.00s) === RUN TestLoadBalancerIDParse --- PASS: TestLoadBalancerIDParse (0.00s) === RUN TestVMScaleSetRenderAzure I0803 22:17:04.112066 15803 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRenderAzure (0.00s) === RUN TestVMScaleSetFind --- PASS: TestVMScaleSetFind (0.00s) === RUN TestVMScaleSetRun I0803 22:17:04.112352 15803 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRun (0.00s) === RUN TestVMScaleSetCheckChanges === RUN TestVMScaleSetCheckChanges/test_case_0 === RUN TestVMScaleSetCheckChanges/test_case_1 === RUN TestVMScaleSetCheckChanges/test_case_2 === RUN TestVMScaleSetCheckChanges/test_case_3 --- PASS: TestVMScaleSetCheckChanges (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_0 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_1 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_2 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_3 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azuretasks 0.031s ? k8s.io/kops/upup/pkg/fi/cloudup/bootstrapchannelbuilder [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/cloudformation [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/do [no test files] === RUN Test_Find === RUN Test_Find/successfully_found_volume === RUN Test_Find/no_volume_found === RUN Test_Find/error_from_server --- PASS: Test_Find (0.00s) --- PASS: Test_Find/successfully_found_volume (0.00s) --- PASS: Test_Find/no_volume_found (0.00s) --- PASS: Test_Find/error_from_server (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/dotasks 0.023s ? k8s.io/kops/upup/pkg/fi/cloudup/gce [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcediscovery [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcemetadata [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmsigner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmverifier [no test files] === RUN TestProjectIAMBinding I0803 22:17:14.785337 18742 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0803 22:17:14.786031 18742 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0803 22:17:14.786236 18742 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0803 22:17:14.786847 18742 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0803 22:17:14.787091 18742 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0803 22:17:14.787459 18742 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestProjectIAMBinding (0.00s) === RUN TestServiceAccount I0803 22:17:14.787879 18742 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0803 22:17:14.788080 18742 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0803 22:17:14.788273 18742 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0803 22:17:14.788441 18742 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0803 22:17:14.788606 18742 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0803 22:17:14.788679 18742 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestServiceAccount (0.00s) === RUN TestStorageBucketIAM I0803 22:17:14.788955 18742 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0803 22:17:14.789345 18742 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0803 22:17:14.789488 18742 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0803 22:17:14.789912 18742 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0803 22:17:14.790075 18742 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0803 22:17:14.790266 18742 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestStorageBucketIAM (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/gcetasks 0.078s ? k8s.io/kops/upup/pkg/fi/cloudup/hetzner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/hetznertasks [no test files] === RUN Test_OpenstackCloud_GetApiIngressStatus === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled --- PASS: Test_OpenstackCloud_GetApiIngressStatus (0.01s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled (0.01s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstack 0.038s === RUN TestFilterPortsReturnsAllPorts --- PASS: TestFilterPortsReturnsAllPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPort --- PASS: TestFilterPortsReturnsOnlyTaggedPort (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPorts --- PASS: TestFilterPortsReturnsOnlyTaggedPorts (0.00s) === RUN Test_Port_GetDependencies --- PASS: Test_Port_GetDependencies (0.00s) === RUN Test_NewPortTaskFromCloud === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found === RUN Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups --- PASS: Test_NewPortTaskFromCloud (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found (0.00s) --- PASS: Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups (0.00s) === RUN Test_Port_Find === RUN Test_Port_Find/nothing_found === RUN Test_Port_Find/port_found_no_tags === RUN Test_Port_Find/port_found_with_tags === RUN Test_Port_Find/multiple_ports_found === RUN Test_Port_Find/error_listing_ports --- PASS: Test_Port_Find (0.00s) --- PASS: Test_Port_Find/nothing_found (0.00s) --- PASS: Test_Port_Find/port_found_no_tags (0.00s) --- PASS: Test_Port_Find/port_found_with_tags (0.00s) --- PASS: Test_Port_Find/multiple_ports_found (0.00s) --- PASS: Test_Port_Find/error_listing_ports (0.00s) === RUN Test_Port_CheckChanges === RUN Test_Port_CheckChanges/actual_nil_all_required_fields_set === RUN Test_Port_CheckChanges/actual_nil_required_field_Name_nil === RUN Test_Port_CheckChanges/actual_nil_required_field_Network_nil === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set --- PASS: Test_Port_CheckChanges (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_all_required_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Name_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Network_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set (0.00s) === RUN Test_Port_RenderOpenstack === RUN Test_Port_RenderOpenstack/actual_not_nil === RUN Test_Port_RenderOpenstack/actual_nil_success === RUN Test_Port_RenderOpenstack/actual_nil_cloud_error --- PASS: Test_Port_RenderOpenstack (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_not_nil (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_success (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_cloud_error (0.00s) === RUN Test_Port_createOptsFromPortTask === RUN Test_Port_createOptsFromPortTask/all_fields_set === RUN Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups --- PASS: Test_Port_createOptsFromPortTask (0.00s) --- PASS: Test_Port_createOptsFromPortTask/all_fields_set (0.00s) --- PASS: Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstacktasks 0.023s ? k8s.io/kops/upup/pkg/fi/cloudup/spotinsttasks [no test files] === RUN TestWriteValue === RUN TestWriteValue/null === RUN TestWriteValue/empty_list === RUN TestWriteValue/string === RUN TestWriteValue/list === RUN TestWriteValue/list_of_objects === RUN TestWriteValue/object_block --- PASS: TestWriteValue (0.00s) --- PASS: TestWriteValue/null (0.00s) --- PASS: TestWriteValue/empty_list (0.00s) --- PASS: TestWriteValue/string (0.00s) --- PASS: TestWriteValue/list (0.00s) --- PASS: TestWriteValue/list_of_objects (0.00s) --- PASS: TestWriteValue/object_block (0.00s) === RUN TestWriteLiteral === RUN TestWriteLiteral/string === RUN TestWriteLiteral/traversal === RUN TestWriteLiteral/provider_alias === RUN TestWriteLiteral/file --- PASS: TestWriteLiteral (0.00s) --- PASS: TestWriteLiteral/string (0.00s) --- PASS: TestWriteLiteral/traversal (0.00s) --- PASS: TestWriteLiteral/provider_alias (0.00s) --- PASS: TestWriteLiteral/file (0.00s) === RUN TestWriteLiteralList === RUN TestWriteLiteralList/empty_list === RUN TestWriteLiteralList/one_literal === RUN TestWriteLiteralList/two_literals === RUN TestWriteLiteralList/one_traversal_literal,_one_string_literal --- PASS: TestWriteLiteralList (0.00s) --- PASS: TestWriteLiteralList/empty_list (0.00s) --- PASS: TestWriteLiteralList/one_literal (0.00s) --- PASS: TestWriteLiteralList/two_literals (0.00s) --- PASS: TestWriteLiteralList/one_traversal_literal,_one_string_literal (0.00s) === RUN TestWriteMap === RUN TestWriteMap/empty_map === RUN TestWriteMap/simple_map === RUN TestWriteMap/complex_keys === RUN TestWriteMap/literal_list --- PASS: TestWriteMap (0.00s) --- PASS: TestWriteMap/empty_map (0.00s) --- PASS: TestWriteMap/simple_map (0.00s) --- PASS: TestWriteMap/complex_keys (0.00s) --- PASS: TestWriteMap/literal_list (0.00s) === RUN TestWriteMapLiterals === RUN TestWriteMapLiterals/literal_values --- PASS: TestWriteMapLiterals (0.00s) --- PASS: TestWriteMapLiterals/literal_values (0.00s) === RUN TestWriteLocalsOutputs === RUN TestWriteLocalsOutputs/empty_map === RUN TestWriteLocalsOutputs/single_output === RUN TestWriteLocalsOutputs/list_output --- PASS: TestWriteLocalsOutputs (0.00s) --- PASS: TestWriteLocalsOutputs/empty_map (0.00s) --- PASS: TestWriteLocalsOutputs/single_output (0.00s) --- PASS: TestWriteLocalsOutputs/list_output (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraform 0.030s === RUN TestGetOutputs === RUN TestGetOutputs/empty_map === RUN TestGetOutputs/single_output === RUN TestGetOutputs/list_output === RUN TestGetOutputs/duplicate_names === RUN TestGetOutputs/duplicate_values --- PASS: TestGetOutputs (0.00s) --- PASS: TestGetOutputs/empty_map (0.00s) --- PASS: TestGetOutputs/single_output (0.00s) --- PASS: TestGetOutputs/list_output (0.00s) --- PASS: TestGetOutputs/duplicate_names (0.00s) --- PASS: TestGetOutputs/duplicate_values (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraformWriter 0.015s === RUN TestKeypairDeps --- PASS: TestKeypairDeps (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/fitasks 0.021s ? k8s.io/kops/upup/pkg/fi/loader [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/cloudinit [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/local [no test files] === RUN TestArchiveDependencies --- PASS: TestArchiveDependencies (0.00s) === RUN TestBindMountCommands I0803 22:17:24.932685 20303 bindmount.go:224] running mount command [mount --bind /containerized_mounter /containerized_mounter] I0803 22:17:24.932740 20303 bindmount.go:233] running mount command [mount -o remount,exec /containerized_mounter] I0803 22:17:24.932752 20303 bindmount.go:224] running mount command [mount --rbind /var/lib/kubelet/ /containerized_mounter/rootfs/var/lib/kubelet] I0803 22:17:24.932763 20303 bindmount.go:244] running mount command [mount --make-rshared /containerized_mounter/rootfs/var/lib/kubelet] I0803 22:17:24.932774 20303 bindmount.go:224] running mount command [mount --bind -o ro /proc /containerized_mounter/rootfs/proc] I0803 22:17:24.932787 20303 bindmount.go:224] running mount command [mount --bind -o ro /dev /containerized_mounter/rootfs/dev] I0803 22:17:24.932798 20303 bindmount.go:224] running mount command [mount --bind -o ro /etc/resolv.conf /containerized_mounter/rootfs/etc/resolv.conf] --- PASS: TestBindMountCommands (0.00s) === RUN TestBindMountDependencies --- PASS: TestBindMountDependencies (0.00s) === RUN TestFileDependencies === RUN TestFileDependencies/user === RUN TestFileDependencies/parentDir === RUN TestFileDependencies/afterFiles --- PASS: TestFileDependencies (0.00s) --- PASS: TestFileDependencies/user (0.00s) --- PASS: TestFileDependencies/parentDir (0.00s) --- PASS: TestFileDependencies/afterFiles (0.00s) === RUN TestIssueCertFileDependencies --- PASS: TestIssueCertFileDependencies (0.00s) === RUN TestLoadImageTask_Deps --- PASS: TestLoadImageTask_Deps (0.00s) === RUN TestServiceTask_Deps --- PASS: TestServiceTask_Deps (0.00s) === RUN TestServiceTask_UnknownTypes W0803 22:17:24.933205 20303 service.go:93] Unhandled type *nodetasks.FakeTask in Service::GetDependencies: &{} --- PASS: TestServiceTask_UnknownTypes (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/nodeup/nodetasks 0.054s ? k8s.io/kops/upup/pkg/fi/nodeup/nodetasks/dnstasks [no test files] ? k8s.io/kops/upup/pkg/fi/secrets [no test files] === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) === RUN TestStringSlicesEqualIgnoreOrder --- PASS: TestStringSlicesEqualIgnoreOrder (0.00s) === RUN TestHashString --- PASS: TestHashString (0.00s) === RUN TestSanitizeString --- PASS: TestSanitizeString (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/utils 0.093s ? k8s.io/kops/upup/pkg/kutil [no test files] ? k8s.io/kops/upup/tools/generators/fitask [no test files] ? k8s.io/kops/util/pkg/architectures [no test files] === RUN TestFindDistribution --- PASS: TestFindDistribution (0.00s) PASS ok k8s.io/kops/util/pkg/distributions 0.003s ? k8s.io/kops/util/pkg/env [no test files] === RUN TestWithTee --- PASS: TestWithTee (0.00s) PASS ok k8s.io/kops/util/pkg/exec 0.010s === RUN TestNewHasher === RUN TestNewHasher/sha256 === RUN TestNewHasher/sha1 === RUN TestNewHasher/md5 --- PASS: TestNewHasher (0.00s) --- PASS: TestNewHasher/sha256 (0.00s) --- PASS: TestNewHasher/sha1 (0.00s) --- PASS: TestNewHasher/md5 (0.00s) === RUN TestMemberFromString === RUN TestMemberFromString/sha256_1 === RUN TestMemberFromString/sha256_2 === RUN TestMemberFromString/sha256_3 === RUN TestMemberFromString/sha1_1 === RUN TestMemberFromString/sha1_2 === RUN TestMemberFromString/sha1_3 === RUN TestMemberFromString/md5_1 === RUN TestMemberFromString/md5_2 === RUN TestMemberFromString/md5_3 --- PASS: TestMemberFromString (0.00s) --- PASS: TestMemberFromString/sha256_1 (0.00s) --- PASS: TestMemberFromString/sha256_2 (0.00s) --- PASS: TestMemberFromString/sha256_3 (0.00s) --- PASS: TestMemberFromString/sha1_1 (0.00s) --- PASS: TestMemberFromString/sha1_2 (0.00s) --- PASS: TestMemberFromString/sha1_3 (0.00s) --- PASS: TestMemberFromString/md5_1 (0.00s) --- PASS: TestMemberFromString/md5_2 (0.00s) --- PASS: TestMemberFromString/md5_3 (0.00s) === RUN TestFromString === RUN TestFromString/sha256 === RUN TestFromString/sha1 === RUN TestFromString/md5 === RUN TestFromString/unknown --- PASS: TestFromString (0.00s) --- PASS: TestFromString/sha256 (0.00s) --- PASS: TestFromString/sha1 (0.00s) --- PASS: TestFromString/md5 (0.00s) --- PASS: TestFromString/unknown (0.00s) PASS ok k8s.io/kops/util/pkg/hashing 0.004s === RUN TestKeys --- PASS: TestKeys (0.00s) === RUN TestSortedKeys --- PASS: TestSortedKeys (0.00s) PASS ok k8s.io/kops/util/pkg/maps 0.003s ? k8s.io/kops/util/pkg/mirrors [no test files] === RUN TestGetProxyEnvVars W0803 22:16:59.601270 14420 proxy.go:34] EgressProxy set but no proxy host provided --- PASS: TestGetProxyEnvVars (0.00s) PASS ok k8s.io/kops/util/pkg/proxy 0.031s === RUN TestSet === RUN TestSet/simple_setting === RUN TestSet/setting_with_wildcard === RUN TestSet/creating_missing_objects === RUN TestSet/set_int === RUN TestSet/set_int32 === RUN TestSet/set_int64 === RUN TestSet/set_int_pointer === RUN TestSet/set_int32_pointer === RUN TestSet/set_int64_pointer === RUN TestSet/set_enum === RUN TestSet/set_enum_slice === RUN TestSet/append_enum_slice --- PASS: TestSet (0.00s) --- PASS: TestSet/simple_setting (0.00s) --- PASS: TestSet/setting_with_wildcard (0.00s) --- PASS: TestSet/creating_missing_objects (0.00s) --- PASS: TestSet/set_int (0.00s) --- PASS: TestSet/set_int32 (0.00s) --- PASS: TestSet/set_int64 (0.00s) --- PASS: TestSet/set_int_pointer (0.00s) --- PASS: TestSet/set_int32_pointer (0.00s) --- PASS: TestSet/set_int64_pointer (0.00s) --- PASS: TestSet/set_enum (0.00s) --- PASS: TestSet/set_enum_slice (0.00s) --- PASS: TestSet/append_enum_slice (0.00s) === RUN TestSetInvalidPath === RUN TestSetInvalidPath/setting_with_wildcard === RUN TestSetInvalidPath/creating_missing_objects === RUN TestSetInvalidPath/set_int === RUN TestSetInvalidPath/set_int32 === RUN TestSetInvalidPath/set_int64 --- PASS: TestSetInvalidPath (0.00s) --- PASS: TestSetInvalidPath/setting_with_wildcard (0.00s) --- PASS: TestSetInvalidPath/creating_missing_objects (0.00s) --- PASS: TestSetInvalidPath/set_int (0.00s) --- PASS: TestSetInvalidPath/set_int32 (0.00s) --- PASS: TestSetInvalidPath/set_int64 (0.00s) === RUN TestUnset === RUN TestUnset/simple_unsetting === RUN TestUnset/unsetting_with_wildcard === RUN TestUnset/uset_int === RUN TestUnset/unset_int32 === RUN TestUnset/unset_int64 === RUN TestUnset/unset_int_pointer === RUN TestUnset/unset_int32_pointer === RUN TestUnset/unset_int64_pointer === RUN TestUnset/unset_enum === RUN TestUnset/unset_enum_slice --- PASS: TestUnset (0.00s) --- PASS: TestUnset/simple_unsetting (0.00s) --- PASS: TestUnset/unsetting_with_wildcard (0.00s) --- PASS: TestUnset/uset_int (0.00s) --- PASS: TestUnset/unset_int32 (0.00s) --- PASS: TestUnset/unset_int64 (0.00s) --- PASS: TestUnset/unset_int_pointer (0.00s) --- PASS: TestUnset/unset_int32_pointer (0.00s) --- PASS: TestUnset/unset_int64_pointer (0.00s) --- PASS: TestUnset/unset_enum (0.00s) --- PASS: TestUnset/unset_enum_slice (0.00s) === RUN TestUnsetInvalidPath === RUN TestUnsetInvalidPath/usetting_with_wildcard === RUN TestUnsetInvalidPath/missing_objects --- PASS: TestUnsetInvalidPath (0.00s) --- PASS: TestUnsetInvalidPath/usetting_with_wildcard (0.00s) --- PASS: TestUnsetInvalidPath/missing_objects (0.00s) === RUN TestParseFieldPath === RUN TestParseFieldPath/test_Spec.Containers.Image === RUN TestParseFieldPath/test_Spec.Containers[0].Image === RUN TestParseFieldPath/test_Spec.Containers[*].Image --- PASS: TestParseFieldPath (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers.Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[0].Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[*].Image (0.00s) PASS ok k8s.io/kops/util/pkg/reflectutils 0.005s === RUN TestGetUniqueStrings --- PASS: TestGetUniqueStrings (0.00s) === RUN TestContains --- PASS: TestContains (0.00s) PASS ok k8s.io/kops/util/pkg/slice 0.003s ? k8s.io/kops/util/pkg/tables [no test files] === RUN TestSplitToSections --- PASS: TestSplitToSections (0.00s) PASS ok k8s.io/kops/util/pkg/text 0.003s ? k8s.io/kops/util/pkg/ui [no test files] === RUN TestAzureBlobPathBase === RUN TestAzureBlobPathBase/Test_case_0 === RUN TestAzureBlobPathBase/Test_case_1 === RUN TestAzureBlobPathBase/Test_case_2 --- PASS: TestAzureBlobPathBase (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_2 (0.00s) === RUN TestAzureBlobPathPath === RUN TestAzureBlobPathPath/Test_case_0 === RUN TestAzureBlobPathPath/Test_case_1 === RUN TestAzureBlobPathPath/Test_case_2 --- PASS: TestAzureBlobPathPath (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_2 (0.00s) === RUN TestAzureBlobPathJoin --- PASS: TestAzureBlobPathJoin (0.00s) === RUN TestBuildVaultPath context_test.go:27: No vault dev token set --- SKIP: TestBuildVaultPath (0.00s) === RUN TestCreateFile --- PASS: TestCreateFile (0.00s) === RUN TestWriteTo --- PASS: TestWriteTo (0.00s) === RUN TestMemFsCreateFile --- PASS: TestMemFsCreateFile (0.00s) === RUN TestMemFsReadDir --- PASS: TestMemFsReadDir (0.00s) === RUN TestMemFsReadTree --- PASS: TestMemFsReadTree (0.00s) === RUN Test_VFSPath --- PASS: Test_VFSPath (0.00s) === RUN Test_S3Path_Parse --- PASS: Test_S3Path_Parse (0.00s) === RUN Test_newVaultPath vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_newVaultPath (0.00s) === RUN Test_newVaultPathHostOnly --- PASS: Test_newVaultPathHostOnly (0.00s) === RUN Test_WriteFileReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileReadFile (0.00s) === RUN Test_WriteFileDeleteFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileDeleteFile (0.00s) === RUN Test_CreateFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_CreateFile (0.00s) === RUN Test_ReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadFile (0.00s) === RUN Test_Join vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_Join (0.00s) === RUN Test_VaultReadDirList vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadDirList (0.00s) === RUN Test_ReadDir vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadDir (0.00s) === RUN Test_IsDirectory vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_IsDirectory (0.00s) === RUN Test_VaultReadTree vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadTree (0.00s) PASS ok k8s.io/kops/util/pkg/vfs 0.024s === RUN TestGSRenderTerraform === RUN TestGSRenderTerraform/gs://foo/bar --- PASS: TestGSRenderTerraform (0.00s) --- PASS: TestGSRenderTerraform/gs://foo/bar (0.00s) === RUN TestS3RenderTerraform === RUN TestS3RenderTerraform/s3://foo/bar I0803 22:17:16.139108 18930 s3context.go:90] Found S3_ENDPOINT="foo.s3.amazonaws.com", using as non-AWS S3 backend W0803 22:17:16.139153 18930 s3context.go:238] Unable to read bucket encryption policy for "foo" in region "us-east-1": will encrypt using AES256 --- PASS: TestS3RenderTerraform (0.00s) --- PASS: TestS3RenderTerraform/s3://foo/bar (0.00s) PASS ok k8s.io/kops/util/pkg/vfs/tests 0.069s >>> kops: Entering fakeroot... >>> kops*: Running postcheck for kops >>> kops*: Preparing package kops... >>> kops*: Stripping binaries >>> kops*: Scanning shared objects >>> kops*: Tracing dependencies... >>> kops*: Package size: 123.1 MB >>> kops*: Compressing data... >>> kops*: Create checksum... >>> kops*: Create kops-1.24.0-r1.apk >>> kops: Build complete at Wed, 03 Aug 2022 22:18:27 +0000 elapsed time 0h 3m 36s >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Uninstalling dependencies... (1/2) Purging .makedepends-kops (20220803.221453) (2/2) Purging kubernetes (1.24.3-r2) OK: 715 MiB in 104 packages >>> kops: Updating the testing/x86_64 repository index... >>> kops: Signing the index...