>>> kops: Building testing/kops 1.24.1-r0 (using abuild 3.9.0-r5) started Thu, 11 Aug 2022 05:28:50 +0000 >>> kops: Checking sanity of /home/buildozer/aports/testing/kops/APKBUILD... >>> kops: Analyzing dependencies... >>> kops: Installing for build: build-base kubernetes go (1/2) Installing kubernetes (1.24.3-r2) (2/2) Installing .makedepends-kops (20220811.052852) OK: 717 MiB in 118 packages >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.24.1.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 146 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 curl: (22) The requested URL returned error: 404 >>> kops: Fetching kops-1.24.1.tar.gz::https://github.com/kubernetes/kops/archive/v1.24.1.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 177k 0 177k 0 0 237k 0 --:--:-- --:--:-- --:--:-- 237k 100 10.2M 0 10.2M 0 0 6020k 0 --:--:-- 0:00:01 --:--:-- 10.0M 100 14.9M 0 14.9M 0 0 5556k 0 --:--:-- 0:00:02 --:--:-- 7540k 100 20.2M 0 20.2M 0 0 5526k 0 --:--:-- 0:00:03 --:--:-- 6842k 100 24.1M 0 24.1M 0 0 5200k 0 --:--:-- 0:00:04 --:--:-- 6126k 100 28.8M 0 28.8M 0 0 5136k 0 --:--:-- 0:00:05 --:--:-- 5866k 100 29.4M 0 29.4M 0 0 5092k 0 --:--:-- 0:00:05 --:--:-- 4704k >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.24.1.tar.gz >>> kops: Checking sha512sums... kops-1.24.1.tar.gz: OK >>> kops: Unpacking /var/cache/distfiles/edge/kops-1.24.1.tar.gz... go: downloading k8s.io/klog/v2 v2.70.1 go: downloading github.com/aws/aws-sdk-go v1.44.60 go: downloading github.com/gophercloud/gophercloud v0.25.0 go: downloading github.com/google/uuid v1.3.0 go: downloading github.com/blang/semver/v4 v4.0.0 go: downloading github.com/cert-manager/cert-manager v1.8.2 go: downloading github.com/prometheus/client_golang v1.12.2 go: downloading go.uber.org/multierr v1.8.0 go: downloading github.com/spf13/pflag v1.0.5 go: downloading k8s.io/apimachinery v0.24.3 go: downloading k8s.io/client-go v0.24.3 go: downloading k8s.io/api v0.24.3 go: downloading github.com/spf13/cobra v1.5.0 go: downloading google.golang.org/api v0.88.0 go: downloading github.com/spf13/viper v1.12.0 go: downloading k8s.io/component-base v0.24.3 go: downloading k8s.io/klog v1.0.0 go: downloading k8s.io/cli-runtime v0.24.3 go: downloading golang.org/x/sys v0.0.0-20220721230656-c6bc011c0c49 go: downloading sigs.k8s.io/controller-runtime v0.12.3 go: downloading sigs.k8s.io/yaml v1.3.0 go: downloading github.com/go-logr/logr v1.2.3 go: downloading github.com/google/go-containerregistry v0.11.0 go: downloading cloud.google.com/go/compute v1.7.0 go: downloading golang.org/x/oauth2 v0.0.0-20220718184931-c8730f7fcb92 go: downloading gopkg.in/gcfg.v1 v1.2.3 go: downloading k8s.io/legacy-cloud-providers v0.24.3 go: downloading github.com/digitalocean/godo v1.81.0 go: downloading github.com/google/go-cmp v0.5.8 go: downloading github.com/aws/amazon-ec2-instance-selector/v2 v2.3.3 go: downloading golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d go: downloading helm.sh/helm/v3 v3.9.2 go: downloading k8s.io/kubectl v0.24.3 go: downloading github.com/stretchr/testify v1.8.0 go: downloading gopkg.in/inf.v0 v0.9.1 go: downloading golang.org/x/net v0.0.0-20220708220712-1185a9018129 go: downloading golang.org/x/sync v0.0.0-20220601150217-0de741cfad7f go: downloading github.com/hetznercloud/hcloud-go v1.35.1 go: downloading github.com/pelletier/go-toml v1.9.5 go: downloading k8s.io/kubelet v0.24.3 go: downloading k8s.io/mount-utils v0.24.3 go: downloading k8s.io/utils v0.0.0-20220713171938-56c0de1e6f5e go: downloading github.com/Azure/azure-pipeline-go v0.2.3 go: downloading github.com/Azure/azure-storage-blob-go v0.15.0 go: downloading github.com/go-ini/ini v1.66.6 go: downloading github.com/hashicorp/vault/api v1.7.2 go: downloading github.com/pkg/sftp v1.13.5 go: downloading go.uber.org/atomic v1.9.0 go: downloading github.com/fsnotify/fsnotify v1.5.4 go: downloading github.com/mitchellh/mapstructure v1.5.0 go: downloading github.com/spf13/afero v1.8.2 go: downloading github.com/spf13/cast v1.5.0 go: downloading github.com/spf13/jwalterweatherman v1.1.0 go: downloading github.com/inconshreveable/mousetrap v1.0.0 go: downloading github.com/prometheus/client_model v0.2.0 go: downloading github.com/prometheus/common v0.32.1 go: downloading github.com/evanphx/json-patch v5.6.0+incompatible go: downloading github.com/gogo/protobuf v1.3.2 go: downloading github.com/google/gofuzz v1.2.0 go: downloading github.com/jacksontj/memberlistmesh v0.0.0-20190905163944-93462b9d2bb7 go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.2.1 go: downloading github.com/weaveworks/mesh v0.0.0-20191105120815-58dbcc3e8e63 go: downloading github.com/sergi/go-diff v1.2.0 go: downloading github.com/apparentlymart/go-cidr v1.1.0 go: downloading github.com/MakeNowJust/heredoc/v2 v2.0.1 go: downloading github.com/davecgh/go-spew v1.1.1 go: downloading github.com/google/gnostic v0.5.7-v3refs go: downloading github.com/imdario/mergo v0.3.12 go: downloading golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 go: downloading github.com/Azure/azure-sdk-for-go v66.0.0+incompatible go: downloading github.com/Azure/go-autorest/autorest/azure/auth v0.5.11 go: downloading github.com/Masterminds/sprig/v3 v3.2.2 go: downloading github.com/Azure/go-autorest/autorest/to v0.4.0 go: downloading gopkg.in/square/go-jose.v2 v2.6.0 go: downloading gopkg.in/yaml.v2 v2.4.0 go: downloading github.com/golang/protobuf v1.5.2 go: downloading golang.org/x/time v0.0.0-20220210224613-90d013bbcef8 go: downloading google.golang.org/protobuf v1.28.0 go: downloading github.com/spotinst/spotinst-sdk-go v1.124.0 go: downloading github.com/MakeNowJust/heredoc v0.0.0-20170808103936-bb23615498cd go: downloading github.com/Azure/go-autorest/autorest v0.11.27 go: downloading github.com/docker/cli v20.10.17+incompatible go: downloading github.com/mitchellh/go-homedir v1.1.0 go: downloading github.com/opencontainers/go-digest v1.0.0 go: downloading github.com/containerd/stargz-snapshotter/estargz v0.12.0 go: downloading github.com/Azure/go-autorest v14.2.0+incompatible go: downloading gopkg.in/warnings.v0 v0.1.2 go: downloading google.golang.org/appengine v1.6.7 go: downloading github.com/google/go-tpm-tools v0.3.8 go: downloading github.com/google/go-tpm v0.3.3 go: downloading github.com/hashicorp/hcl/v2 v2.13.0 go: downloading github.com/zclconf/go-cty v1.10.0 go: downloading k8s.io/gengo v0.0.0-20220613173612-397b4ae3bce7 go: downloading github.com/google/go-querystring v1.1.0 go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.2 go: downloading github.com/GoogleCloudPlatform/k8s-cloud-provider v1.16.1-0.20210702024009-ea6160c1d0e3 go: downloading k8s.io/cloud-provider v0.24.3 go: downloading github.com/pkg/errors v0.9.1 go: downloading github.com/pmezard/go-difflib v1.0.0 go: downloading gopkg.in/yaml.v3 v3.0.1 go: downloading github.com/chai2010/gettext-go v0.0.0-20160711120539-c6fed771bfd5 go: downloading github.com/mitchellh/go-wordwrap v1.0.0 go: downloading github.com/russross/blackfriday v1.6.0 go: downloading google.golang.org/grpc v1.47.0 go: downloading github.com/googleapis/gax-go/v2 v2.4.0 go: downloading k8s.io/csi-translation-lib v0.24.3 go: downloading github.com/mattn/go-ieproxy v0.0.1 go: downloading github.com/cenkalti/backoff/v3 v3.0.0 go: downloading github.com/hashicorp/errwrap v1.1.0 go: downloading github.com/hashicorp/go-cleanhttp v0.5.2 go: downloading github.com/hashicorp/go-multierror v1.1.1 go: downloading github.com/hashicorp/go-retryablehttp v0.6.6 go: downloading github.com/hashicorp/go-rootcerts v1.0.2 go: downloading github.com/hashicorp/go-secure-stdlib/parseutil v0.1.6 go: downloading github.com/hashicorp/hcl v1.0.0 go: downloading github.com/hashicorp/vault/sdk v0.5.1 go: downloading github.com/hashicorp/go-hclog v1.2.0 go: downloading github.com/kr/fs v0.1.0 go: downloading github.com/Azure/go-autorest/autorest/adal v0.9.18 go: downloading github.com/subosito/gotenv v1.3.0 go: downloading gopkg.in/ini.v1 v1.66.6 go: downloading github.com/magiconair/properties v1.8.6 go: downloading github.com/pelletier/go-toml/v2 v2.0.1 go: downloading github.com/prometheus/procfs v0.7.3 go: downloading github.com/beorn7/perks v1.0.1 go: downloading github.com/cespare/xxhash/v2 v2.1.2 go: downloading golang.org/x/text v0.3.7 go: downloading sigs.k8s.io/json v0.0.0-20211208200746-9f7c6b3444d2 go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de go: downloading sigs.k8s.io/kustomize/api v0.11.4 go: downloading sigs.k8s.io/kustomize/kyaml v0.13.6 go: downloading k8s.io/kube-openapi v0.0.0-20220328201542-3ee0da9b0b42 go: downloading github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 go: downloading github.com/peterbourgon/diskv v2.0.1+incompatible go: downloading k8s.io/apiextensions-apiserver v0.24.2 go: downloading sigs.k8s.io/gateway-api v0.4.1 go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da go: downloading gomodules.xyz/jsonpatch/v2 v2.2.0 go: downloading github.com/hashicorp/go-sockaddr v1.0.2 go: downloading github.com/hashicorp/memberlist v0.3.1 go: downloading github.com/oklog/ulid v1.3.1 go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 go: downloading github.com/Masterminds/goutils v1.1.1 go: downloading github.com/Masterminds/semver/v3 v3.1.1 go: downloading github.com/huandu/xstrings v1.3.2 go: downloading github.com/mitchellh/copystructure v1.2.0 go: downloading github.com/onsi/ginkgo v1.16.5 go: downloading github.com/onsi/gomega v1.18.1 go: downloading go.uber.org/goleak v1.1.12 go: downloading github.com/go-test/deep v1.0.3 go: downloading gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c go: downloading github.com/go-logr/zapr v1.2.0 go: downloading go.uber.org/zap v1.19.1 go: downloading github.com/frankban/quicktest v1.14.3 go: downloading github.com/shopspring/decimal v1.2.0 go: downloading github.com/json-iterator/go v1.1.12 go: downloading github.com/klauspost/compress v1.15.8 go: downloading github.com/vbatts/tar-split v0.11.2 go: downloading github.com/docker/distribution v2.8.1+incompatible go: downloading github.com/Azure/go-autorest/autorest/azure/cli v0.4.5 go: downloading github.com/Azure/go-autorest/logger v0.2.1 go: downloading github.com/dimchansky/utfbom v1.1.1 go: downloading github.com/Azure/go-autorest/tracing v0.6.0 go: downloading github.com/Azure/go-autorest/autorest/mocks v0.4.2 go: downloading github.com/russross/blackfriday/v2 v2.1.0 go: downloading github.com/patrickmn/go-cache v2.1.0+incompatible go: downloading golang.org/x/tools v0.1.8 go: downloading github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6 go: downloading go.opencensus.io v0.23.0 go: downloading golang.org/x/xerrors v0.0.0-20220609144429-65e65417b02f go: downloading github.com/stretchr/objx v0.4.0 go: downloading google.golang.org/genproto v0.0.0-20220624142145-8cd45d7dbd1f go: downloading github.com/hashicorp/go-secure-stdlib/strutil v0.1.2 go: downloading github.com/fatih/color v1.13.0 go: downloading github.com/mattn/go-colorable v0.1.12 go: downloading github.com/mattn/go-isatty v0.0.14 go: downloading github.com/Azure/go-autorest/autorest/date v0.3.0 go: downloading github.com/golang-jwt/jwt/v4 v4.2.0 go: downloading github.com/apparentlymart/go-textseg/v13 v13.0.0 go: downloading github.com/kylelemons/godebug v1.1.0 go: downloading github.com/agext/levenshtein v1.2.1 go: downloading github.com/kr/pretty v0.3.0 go: downloading github.com/mitchellh/go-testing-interface v1.0.0 go: downloading github.com/jmespath/go-jmespath v0.4.0 go: downloading github.com/google/btree v1.0.1 go: downloading github.com/docker/docker v20.10.17+incompatible go: downloading gotest.tools/v3 v3.0.3 go: downloading github.com/sirupsen/logrus v1.9.0 go: downloading github.com/google/go-attestation v0.4.4-0.20220404204839-8820d49b18d9 go: downloading github.com/armon/go-metrics v0.3.10 go: downloading github.com/hashicorp/go-msgpack v0.5.3 go: downloading github.com/miekg/dns v1.1.48 go: downloading github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 go: downloading github.com/mitchellh/reflectwalk v1.0.2 go: downloading github.com/opencontainers/image-spec v1.0.3-0.20220114050600-8b9d41f48198 go: downloading github.com/containerd/containerd v1.6.6 go: downloading oras.land/oras-go v1.2.0 go: downloading github.com/distribution/distribution/v3 v3.0.0-20220526142353-ffbd94cbe269 go: downloading github.com/phayes/freeport v0.0.0-20220201140144-74d24b5ae9f5 go: downloading github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d go: downloading github.com/modern-go/reflect2 v1.0.2 go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd go: downloading github.com/googleapis/enterprise-certificate-proxy v0.1.0 go: downloading github.com/creack/pty v1.1.11 go: downloading github.com/golang/snappy v0.0.4 go: downloading github.com/pierrec/lz4 v2.5.2+incompatible go: downloading github.com/hashicorp/go-plugin v1.4.3 go: downloading github.com/hashicorp/go-secure-stdlib/mlock v0.1.1 go: downloading github.com/hashicorp/go-uuid v1.0.2 go: downloading github.com/hashicorp/go-version v1.2.1 go: downloading github.com/hashicorp/golang-lru v0.5.4 go: downloading github.com/armon/go-radix v1.0.0 go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 go: downloading github.com/go-openapi/swag v0.19.14 go: downloading github.com/ryanuber/go-glob v1.0.0 go: downloading github.com/docker/docker-credential-helpers v0.6.4 go: downloading github.com/kr/text v0.2.0 go: downloading github.com/rogpeppe/go-internal v1.6.2 go: downloading github.com/hashicorp/go-immutable-radix v1.3.1 go: downloading github.com/pascaldekloe/goe v0.1.0 go: downloading github.com/jmespath/go-jmespath/internal/testify v1.5.1 go: downloading github.com/docker/go-connections v0.4.0 go: downloading github.com/moby/spdystream v0.2.0 go: downloading github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 go: downloading github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153 go: downloading github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 go: downloading github.com/go-openapi/jsonreference v0.19.5 go: downloading github.com/nxadm/tail v1.4.8 go: downloading github.com/google/go-tspi v0.2.1-0.20190423175329-115dea689aad go: downloading github.com/fatih/structs v1.1.0 go: downloading github.com/emicklei/go-restful v2.9.5+incompatible go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: downloading github.com/go-errors/errors v1.0.1 go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 go: downloading github.com/xlab/treeprint v0.0.0-20181112141820-a009c3971eca go: downloading github.com/mailru/easyjson v0.7.6 go: downloading github.com/hashicorp/yamux v0.0.0-20180604194846-3520598351bb go: downloading github.com/oklog/run v1.0.0 go: downloading github.com/jhump/protoreflect v1.6.1 go: downloading github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d go: downloading github.com/bshuster-repo/logrus-logstash-hook v1.0.0 go: downloading github.com/bugsnag/bugsnag-go v0.0.0-20141110184014-b1d153021fcd go: downloading github.com/docker/go-metrics v0.0.1 go: downloading github.com/docker/libtrust v0.0.0-20150114040149-fa567046d9b1 go: downloading github.com/gorilla/handlers v1.5.1 go: downloading github.com/yvasiyarov/gorelic v0.0.0-20141212073537-a9bba5b9ab50 go: downloading github.com/PuerkitoBio/purell v1.1.1 go: downloading github.com/go-openapi/jsonpointer v0.19.5 go: downloading gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 go: downloading github.com/google/certificate-transparency-go v1.1.2 go: downloading github.com/josharian/intern v1.0.0 go: downloading github.com/gorilla/mux v1.8.0 go: downloading github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c go: downloading github.com/gomodule/redigo v1.8.2 go: downloading github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0 go: downloading github.com/felixge/httpsnoop v1.0.1 go: downloading github.com/yvasiyarov/go-metrics v0.0.0-20140926110328-57bccd1ccd43 go: downloading github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f go: downloading github.com/moby/locker v1.0.1 go: downloading go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 go: downloading github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 go: downloading github.com/bugsnag/osext v0.0.0-20130617224835-0dd3f918b21b go: downloading github.com/moby/sys/mountinfo v0.5.0 go: downloading github.com/Microsoft/hcsshim v0.9.3 go: downloading golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4 go: downloading github.com/docker/go-units v0.4.0 go: downloading github.com/morikuni/aec v1.0.0 go: downloading github.com/Microsoft/go-winio v0.5.2 go: downloading github.com/containerd/cgroups v1.0.3 go: downloading github.com/Azure/go-autorest/autorest/validation v0.3.1 /bin/sh: tools/get_version.sh: not found mkdir -p /home/buildozer/aports/testing/kops/src/kops-1.24.1/.build/dist/linux/amd64 GOOS=linux GOARCH=amd64 go build "-trimpath" -o /home/buildozer/aports/testing/kops/src/kops-1.24.1/.build/dist/linux/amd64/kops -ldflags=all=" -X k8s.io/kops.Version= -X k8s.io/kops.GitVersion=v20220809-101-g07aaff2d18" k8s.io/kops/cmd/kops go test -v ./... ? k8s.io/kops [no test files] ? k8s.io/kops/channels/cmd/channels [no test files] ? k8s.io/kops/channels/pkg/api [no test files] === RUN Test_Filtering --- PASS: Test_Filtering (0.00s) === RUN Test_Replacement --- PASS: Test_Replacement (0.00s) === RUN Test_GetRequiredUpdates --- PASS: Test_GetRequiredUpdates (0.00s) === RUN Test_NeedsRollingUpdate I0811 05:31:30.823619 34482 addon.go:229] addon test wants to update all nodes I0811 05:31:30.824928 34482 addon.go:229] addon test wants to update worker nodes I0811 05:31:30.825171 34482 addon.go:229] addon test wants to update control-plane nodes --- PASS: Test_NeedsRollingUpdate (0.00s) === RUN Test_InstallPKI I0811 05:31:30.825440 34482 addon.go:262] installing PKI for "test" I0811 05:31:31.648744 34482 addon.go:262] installing PKI for "test" --- PASS: Test_InstallPKI (1.01s) === RUN Test_IsPKIInstalled --- PASS: Test_IsPKIInstalled (0.00s) PASS ok k8s.io/kops/channels/pkg/channels 1.057s === RUN TestGetUpdates --- PASS: TestGetUpdates (0.00s) PASS ok k8s.io/kops/channels/pkg/cmd 0.041s ? k8s.io/kops/cloudmock/aws/mockautoscaling [no test files] ? k8s.io/kops/cloudmock/aws/mockec2 [no test files] ? k8s.io/kops/cloudmock/aws/mockelb [no test files] ? k8s.io/kops/cloudmock/aws/mockelbv2 [no test files] ? k8s.io/kops/cloudmock/aws/mockeventbridge [no test files] ? k8s.io/kops/cloudmock/aws/mockiam [no test files] ? k8s.io/kops/cloudmock/aws/mockroute53 [no test files] ? k8s.io/kops/cloudmock/aws/mocksqs [no test files] ? k8s.io/kops/cloudmock/gce [no test files] ? k8s.io/kops/cloudmock/gce/gcphttp [no test files] ? k8s.io/kops/cloudmock/gce/mockcloudresourcemanager [no test files] ? k8s.io/kops/cloudmock/gce/mockcompute [no test files] ? k8s.io/kops/cloudmock/gce/mockdns [no test files] ? k8s.io/kops/cloudmock/gce/mockiam [no test files] ? k8s.io/kops/cloudmock/gce/mockstorage [no test files] ? k8s.io/kops/cloudmock/openstack [no test files] ? k8s.io/kops/cloudmock/openstack/mockblockstorage [no test files] ? k8s.io/kops/cloudmock/openstack/mockcompute [no test files] ? k8s.io/kops/cloudmock/openstack/mockdns [no test files] ? k8s.io/kops/cloudmock/openstack/mockimage [no test files] ? k8s.io/kops/cloudmock/openstack/mockloadbalancer [no test files] ? k8s.io/kops/cloudmock/openstack/mocknetworking [no test files] === RUN TestCreateClusterMinimal I0811 05:32:01.683250 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:01.683505 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:01.683551 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:01.683585 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:01.683642 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:01.683751 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:01.683765 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:01.683804 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:01.683843 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:01.683919 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:01.683955 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:01.718529 40506 new_cluster.go:1168] Cloud Provider ID = aws I0811 05:32:01.718657 40506 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0811 05:32:01.719176 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:01.733437 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:01.733602 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:01.733645 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:01.733685 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:01.733719 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:01.733746 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:01.733780 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:01.733807 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:01.733827 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:01.733895 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:01.733923 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:01.773080 40506 new_cluster.go:1168] Cloud Provider ID = aws I0811 05:32:01.773155 40506 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0811 05:32:01.773377 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:01.777558 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:01.777683 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:01.777718 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:01.777741 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:01.777765 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:01.777783 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:01.777796 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:01.777812 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:01.777825 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:01.777868 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:01.777898 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:01.806124 40506 new_cluster.go:1168] Cloud Provider ID = aws I0811 05:32:01.806176 40506 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0811 05:32:01.806334 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:01.810286 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:01.810398 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:01.810417 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:01.810446 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:01.810467 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:01.810485 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:01.810499 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:01.810535 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:01.810560 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:01.810595 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:01.810624 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:01.865079 40506 new_cluster.go:1168] Cloud Provider ID = aws I0811 05:32:01.865138 40506 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0811 05:32:01.865320 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:01.872567 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:01.872739 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:01.872780 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:01.872808 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:01.872839 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:01.872863 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:01.872880 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:01.872902 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:01.872919 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:01.872984 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:01.873007 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:01.895186 40506 new_cluster.go:1168] Cloud Provider ID = aws I0811 05:32:01.895246 40506 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0811 05:32:01.895492 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:01.901068 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:01.901179 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:01.901203 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:01.901225 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:01.901251 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:01.901272 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:01.901291 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:01.901311 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:01.901326 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:01.901369 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:01.901393 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:01.956397 40506 new_cluster.go:1168] Cloud Provider ID = aws I0811 05:32:01.956449 40506 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0811 05:32:01.956654 40506 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterMinimal (0.28s) === RUN TestCreateClusterOverride I0811 05:32:01.962417 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:01.962562 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:01.962589 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:01.962615 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:01.962643 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:01.962692 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:01.962709 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:01.962730 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:01.962746 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:01.962825 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:01.962851 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:01.986596 40506 new_cluster.go:1168] Cloud Provider ID = aws I0811 05:32:01.986951 40506 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0811 05:32:01.987163 40506 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterOverride (0.03s) === RUN TestCreateClusterComplex I0811 05:32:01.993901 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:01.994063 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:01.994089 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:01.994111 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:01.994146 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:01.994185 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:01.994198 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:01.994215 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:01.994228 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:01.994276 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:01.994295 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:02.024137 40506 new_cluster.go:1168] Cloud Provider ID = aws I0811 05:32:02.024192 40506 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0811 05:32:02.024433 40506 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterComplex (0.03s) === RUN TestCreateClusterHA I0811 05:32:02.028422 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.028531 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.028564 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.028584 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.028608 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.028655 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.028668 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.028682 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.028704 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.028742 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:02.028771 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:02.087946 40506 new_cluster.go:1168] Cloud Provider ID = aws I0811 05:32:02.088018 40506 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0811 05:32:02.088029 40506 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0811 05:32:02.088038 40506 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0811 05:32:02.088322 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:02.097956 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.098200 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.098252 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.098303 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.098356 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.098398 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.098427 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.098463 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.098490 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.098583 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:02.098626 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:02.205097 40506 new_cluster.go:1168] Cloud Provider ID = aws I0811 05:32:02.205180 40506 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0811 05:32:02.205194 40506 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0811 05:32:02.205204 40506 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0811 05:32:02.205510 40506 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHA (0.19s) === RUN TestCreateClusterHAGCE I0811 05:32:02.214786 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.214932 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.214963 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.214989 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.215019 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.215073 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.215090 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.215112 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.215130 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.215185 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:02.215209 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:02.260091 40506 new_cluster.go:1168] Cloud Provider ID = gce I0811 05:32:02.260166 40506 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterHAGCE (0.06s) === RUN TestCreateClusterGCE I0811 05:32:02.271500 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.271675 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.271708 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.271735 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.271766 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.271818 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.271836 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.271860 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.271878 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.271935 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:02.271960 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:02.304877 40506 new_cluster.go:425] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0811 05:32:02.304927 40506 new_cluster.go:1168] Cloud Provider ID = gce I0811 05:32:02.304964 40506 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterGCE (0.04s) === RUN TestCreateClusterHASharedZone I0811 05:32:02.310650 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.310769 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.310796 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.310820 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.310845 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.310893 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.310911 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.310932 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.310948 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.311007 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:02.311033 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0811 05:32:02.374845 40506 new_cluster.go:781] Running with masters in the same AZs; redundancy will be reduced I0811 05:32:02.374889 40506 new_cluster.go:1168] Cloud Provider ID = aws I0811 05:32:02.374946 40506 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0811 05:32:02.375191 40506 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZone (0.07s) === RUN TestCreateClusterHASharedZones I0811 05:32:02.385305 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.385530 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.385575 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.385619 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.385668 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.385797 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.385844 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.385888 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.385916 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.385998 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:02.386039 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0811 05:32:02.439644 40506 new_cluster.go:781] Running with masters in the same AZs; redundancy will be reduced I0811 05:32:02.439716 40506 new_cluster.go:1168] Cloud Provider ID = aws I0811 05:32:02.439789 40506 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0811 05:32:02.439808 40506 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0811 05:32:02.440135 40506 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZones (0.07s) === RUN TestCreateClusterPrivate I0811 05:32:02.452464 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.452598 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.452626 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.452651 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.452679 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.452726 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.452745 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.452768 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.452785 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.452834 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:02.452861 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:02.502874 40506 new_cluster.go:1168] Cloud Provider ID = aws I0811 05:32:02.502936 40506 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0811 05:32:02.502951 40506 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0811 05:32:02.503176 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:02.503216 40506 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterPrivate (0.06s) === RUN TestCreateClusterPrivateGCE I0811 05:32:02.512296 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.512512 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.512554 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.512585 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.512622 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.512684 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.512714 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.512742 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.512772 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.512839 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:02.512867 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:02.602460 40506 new_cluster.go:425] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0811 05:32:02.602525 40506 new_cluster.go:1168] Cloud Provider ID = gce I0811 05:32:02.602578 40506 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterPrivateGCE (0.10s) === RUN TestCreateClusterWithNGWSpecified I0811 05:32:02.609841 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.609996 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.610028 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.610055 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.610085 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.610136 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.610155 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.610178 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.610194 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.610255 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:02.610280 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:02.642405 40506 new_cluster.go:1168] Cloud Provider ID = aws I0811 05:32:02.642514 40506 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0811 05:32:02.642543 40506 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0811 05:32:02.642948 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:02.642998 40506 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithNGWSpecified (0.04s) === RUN TestCreateClusterWithINGWSpecified I0811 05:32:02.654642 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.654878 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.654925 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.654974 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.655024 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.655101 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.655135 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.655170 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.655198 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.655280 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:02.655326 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:02.726109 40506 new_cluster.go:1168] Cloud Provider ID = aws I0811 05:32:02.726162 40506 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0811 05:32:02.726176 40506 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0811 05:32:02.726388 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:02.726415 40506 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithINGWSpecified (0.08s) === RUN TestCreateClusterSharedVPC I0811 05:32:02.732665 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.732789 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.732818 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.732841 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.732866 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.732908 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.732926 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.732945 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.732958 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.733001 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:02.733027 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:02.758588 40506 new_cluster.go:1168] Cloud Provider ID = aws I0811 05:32:02.758633 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0811 05:32:02.758684 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0811 05:32:02.758741 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0811 05:32:02.758766 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0811 05:32:02.758802 40506 subnets.go:185] Assigned CIDR 10.2.0.0/15 to subnet us-test-1a I0811 05:32:02.758995 40506 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedVPC (0.03s) === RUN TestCreateClusterSharedSubnets I0811 05:32:02.766351 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.766540 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.766575 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.766607 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.766639 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.766693 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.766710 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.766733 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.766751 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.766849 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:02.766877 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:02.797340 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0811 05:32:02.797376 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0811 05:32:02.797445 40506 new_cluster.go:1168] Cloud Provider ID = aws I0811 05:32:02.797493 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0811 05:32:02.797514 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0811 05:32:02.797539 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0811 05:32:02.797556 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0811 05:32:02.797933 40506 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnets (0.04s) === RUN TestCreateClusterSharedSubnetsVpcLookup I0811 05:32:02.803627 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.803751 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.803778 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.803803 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.803831 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.803881 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.803901 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.803921 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.803935 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.803983 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:02.804007 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:02.889025 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:32:02.889064 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0811 05:32:02.889086 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0811 05:32:02.889146 40506 new_cluster.go:1168] Cloud Provider ID = aws I0811 05:32:02.889186 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0811 05:32:02.889207 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0811 05:32:02.889233 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0811 05:32:02.889250 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0811 05:32:02.889510 40506 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnetsVpcLookup (0.09s) === RUN TestCreateClusterPrivateSharedSubnets I0811 05:32:02.895301 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.895447 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.895475 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.895501 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.895535 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.895587 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.895605 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.895628 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.895643 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.895710 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:02.895735 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:02.995244 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0811 05:32:02.995294 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0811 05:32:02.995366 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0811 05:32:02.995386 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0811 05:32:02.995416 40506 new_cluster.go:1168] Cloud Provider ID = aws I0811 05:32:02.995468 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0811 05:32:02.995488 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0811 05:32:02.995512 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0811 05:32:02.995526 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0811 05:32:02.995886 40506 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterPrivateSharedSubnets (0.11s) === RUN TestCreateClusterIPv6 I0811 05:32:03.001977 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:03.002138 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:03.002168 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:03.002194 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:03.002224 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:03.002278 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:03.002295 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:03.002319 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:03.002337 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:03.002410 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:03.002437 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:03.034356 40506 new_cluster.go:1168] Cloud Provider ID = aws I0811 05:32:03.034418 40506 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0811 05:32:03.034717 40506 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterIPv6 (0.04s) === RUN TestCreateClusterKarpenter I0811 05:32:03.040574 40506 featureflag.go:162] FeatureFlag "Karpenter"=true I0811 05:32:03.040586 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:03.040707 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:03.040736 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:03.040761 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:03.040789 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:03.040833 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:03.040851 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:03.040873 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:03.040887 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:03.040936 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:03.040961 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0811 05:32:03.088179 40506 new_cluster.go:1168] Cloud Provider ID = aws I0811 05:32:03.088249 40506 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0811 05:32:03.088261 40506 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0811 05:32:03.088271 40506 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0811 05:32:03.088552 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:03.097672 40506 featureflag.go:162] FeatureFlag "Karpenter"=false --- PASS: TestCreateClusterKarpenter (0.06s) === RUN TestParseCloudLabels --- PASS: TestParseCloudLabels (0.00s) === RUN TestConfirmation --- PASS: TestConfirmation (0.00s) === RUN TestMinimal I0811 05:32:03.097973 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:03.098131 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:03.098162 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:03.098187 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:03.098214 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:03.098263 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:03.098281 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:03.098306 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:03.098323 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:03.111869 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:03.113437 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:03.113514 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0811 05:32:04.062204 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:05.692034 40506 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0811 05:32:05.695428 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:05.697979 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:05.698014 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:05.698036 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:05.698135 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0811 05:32:05.698168 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:05.698537 40506 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0811 05:32:05.704163 40506 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0811 05:32:05.704879 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:05.705846 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:05.706376 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:05.709117 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:05.709190 40506 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0811 05:32:05.709889 40506 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0811 05:32:05.755216 40506 target.go:112] Terraform output is in /tmp/test2029069864/out I0811 05:32:05.756987 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:05.757095 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:05.757131 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:05.757160 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:05.757187 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:05.757213 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:05.757236 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:05.757257 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:05.757272 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:05.767591 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:05.768849 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:05.768896 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0811 05:32:05.803777 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0811 05:32:05.876648 40506 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0811 05:32:05.877494 40506 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0811 05:32:05.877544 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:05.877655 40506 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0811 05:32:05.877697 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:05.877986 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:05.878035 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0811 05:32:05.878215 40506 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0811 05:32:05.878251 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:05.878403 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } I0811 05:32:05.881632 40506 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0811 05:32:05.884716 40506 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0811 05:32:05.885043 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:05.885268 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:05.887101 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:05.888502 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:05.888648 40506 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0811 05:32:05.888986 40506 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0811 05:32:05.893556 40506 target.go:168] Cloudformation output is in /tmp/test2214259375/out --- PASS: TestMinimal (2.81s) === RUN TestMinimal_v1_23 I0811 05:32:05.903675 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:05.903853 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:05.903908 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:05.903954 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:05.904002 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:05.904149 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:05.904224 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:05.904300 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:05.904375 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:05.920378 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* I0811 05:32:05.922480 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:05.922547 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:05.961231 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:06.119726 40506 executor.go:111] Tasks: 0 done / 90 total; 45 can run W0811 05:32:06.120247 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:06.120263 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:06.120489 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:06.120532 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:06.120661 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:06.120780 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:06.122683 40506 executor.go:111] Tasks: 45 done / 90 total; 19 can run I0811 05:32:06.130194 40506 executor.go:111] Tasks: 64 done / 90 total; 24 can run I0811 05:32:06.130512 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:06.130709 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:06.131652 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:06.133344 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:06.133460 40506 executor.go:111] Tasks: 88 done / 90 total; 2 can run I0811 05:32:06.133756 40506 executor.go:111] Tasks: 90 done / 90 total; 0 can run I0811 05:32:06.187193 40506 target.go:112] Terraform output is in /tmp/test614879994/out I0811 05:32:06.191288 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:06.191497 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:06.191552 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:06.191604 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:06.191656 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:06.191697 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:06.191728 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:06.191765 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:06.191794 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:06.217073 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:06.220530 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:06.220637 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0811 05:32:06.257637 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0811 05:32:06.342736 40506 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0811 05:32:06.343322 40506 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0811 05:32:06.343349 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:06.343443 40506 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0811 05:32:06.343541 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:06.343711 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:06.343799 40506 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key W0811 05:32:06.343833 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:06.343810 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:06.343881 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } I0811 05:32:06.348173 40506 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0811 05:32:06.353040 40506 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0811 05:32:06.353232 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:06.353433 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:06.354245 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:06.355225 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:06.355341 40506 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0811 05:32:06.355564 40506 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0811 05:32:06.357992 40506 target.go:168] Cloudformation output is in /tmp/test2564635136/out --- PASS: TestMinimal_v1_23 (0.46s) === RUN TestMinimal_v1_24 I0811 05:32:06.365823 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:06.365994 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:06.366032 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:06.366100 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:06.366142 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:06.366197 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:06.366220 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:06.366248 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:06.366269 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:06.384466 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0811 05:32:06.386351 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:06.386429 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:06.424041 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:06.557138 40506 executor.go:111] Tasks: 0 done / 91 total; 46 can run I0811 05:32:06.557325 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:06.557348 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:06.557409 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0811 05:32:06.557444 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:06.559681 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:06.560289 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:06.563106 40506 executor.go:111] Tasks: 46 done / 91 total; 19 can run I0811 05:32:06.567820 40506 executor.go:111] Tasks: 65 done / 91 total; 24 can run I0811 05:32:06.568031 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:06.568228 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:06.569245 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:06.570304 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:06.570424 40506 executor.go:111] Tasks: 89 done / 91 total; 2 can run I0811 05:32:06.570746 40506 executor.go:111] Tasks: 91 done / 91 total; 0 can run I0811 05:32:06.614773 40506 target.go:112] Terraform output is in /tmp/test2684144792/out I0811 05:32:06.617438 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:06.617579 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:06.617621 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:06.617656 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:06.617692 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:06.617724 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:06.617742 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:06.617765 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:06.617786 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:06.631493 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:06.633184 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:06.633252 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0811 05:32:06.667931 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0811 05:32:06.758763 40506 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0811 05:32:06.759259 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:06.759368 40506 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0811 05:32:06.759385 40506 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0811 05:32:06.759378 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:06.759413 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:06.759442 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0811 05:32:06.759507 40506 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0811 05:32:06.759554 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0811 05:32:06.759603 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:06.761269 40506 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0811 05:32:06.766437 40506 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0811 05:32:06.766622 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:06.766861 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:06.768686 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:06.771140 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:06.771245 40506 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0811 05:32:06.771465 40506 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0811 05:32:06.773263 40506 target.go:168] Cloudformation output is in /tmp/test2720586319/out --- PASS: TestMinimal_v1_24 (0.41s) === RUN TestNvidia I0811 05:32:06.778641 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:06.778785 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:06.778820 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:06.778852 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:06.778887 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:06.778941 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:06.778960 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:06.778985 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:06.779003 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:06.793584 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:06.795285 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:06.795350 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:06.831549 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:06.912005 40506 executor.go:111] Tasks: 0 done / 86 total; 44 can run W0811 05:32:06.912431 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:06.912612 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:06.912643 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:06.912725 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0811 05:32:06.912819 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:06.912925 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:06.918168 40506 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0811 05:32:06.921579 40506 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0811 05:32:06.921803 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:06.921882 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:06.922563 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:06.923214 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:06.923283 40506 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0811 05:32:06.923481 40506 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0811 05:32:06.961553 40506 target.go:112] Terraform output is in /tmp/test1676888163/out I0811 05:32:06.966047 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:06.966372 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:06.966438 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:06.966507 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:06.966618 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:06.966730 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:06.966878 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:06.967322 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:06.967361 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:06.991965 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:06.994835 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:06.994933 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:07.032169 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0811 05:32:07.129867 40506 executor.go:111] Tasks: 0 done / 86 total; 44 can run W0811 05:32:07.130575 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:07.130585 40506 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0811 05:32:07.130638 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:07.130779 40506 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0811 05:32:07.130705 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:07.130816 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0811 05:32:07.130930 40506 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0811 05:32:07.130959 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:07.131080 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0811 05:32:07.132579 40506 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0811 05:32:07.138830 40506 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0811 05:32:07.139172 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:07.139377 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:07.140441 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:07.140530 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:07.140609 40506 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0811 05:32:07.140778 40506 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0811 05:32:07.143699 40506 target.go:168] Cloudformation output is in /tmp/test1622166544/out --- PASS: TestNvidia (0.38s) === RUN TestMinimalGossip I0811 05:32:07.154407 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:07.154647 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:07.154704 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:07.154760 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:07.154818 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:07.154910 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:07.154943 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:07.154985 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:07.155018 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:07.182162 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:07.182262 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:07.217858 40506 apply_cluster.go:467] Gossip DNS: skipping DNS validation I0811 05:32:07.302728 40506 executor.go:111] Tasks: 0 done / 84 total; 42 can run I0811 05:32:07.302960 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:07.303330 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:07.303647 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:07.305994 40506 executor.go:111] Tasks: 42 done / 84 total; 19 can run I0811 05:32:07.309417 40506 executor.go:111] Tasks: 61 done / 84 total; 21 can run I0811 05:32:07.309575 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:07.309994 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:07.312020 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:07.314105 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:07.314196 40506 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0811 05:32:07.314548 40506 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0811 05:32:07.356153 40506 target.go:112] Terraform output is in /tmp/test4144612190/out --- PASS: TestMinimalGossip (0.20s) === RUN TestMinimalGossipIRSA I0811 05:32:07.358642 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:07.358804 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:07.358848 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:07.358888 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:07.358924 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:07.358989 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:07.359009 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:07.359047 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:07.359067 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:07.378644 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:07.378719 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:07.414443 40506 apply_cluster.go:467] Gossip DNS: skipping DNS validation I0811 05:32:07.487777 40506 executor.go:111] Tasks: 0 done / 87 total; 44 can run I0811 05:32:07.488078 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:07.488309 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:07.488497 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:07.494155 40506 executor.go:111] Tasks: 44 done / 87 total; 20 can run I0811 05:32:07.497101 40506 executor.go:111] Tasks: 64 done / 87 total; 21 can run I0811 05:32:07.497273 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:07.497547 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:07.498179 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:07.498679 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:07.498748 40506 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0811 05:32:07.498932 40506 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0811 05:32:07.534902 40506 target.go:112] Terraform output is in /tmp/test696834954/out --- PASS: TestMinimalGossipIRSA (0.18s) === RUN TestMinimalGCE I0811 05:32:07.539044 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:07.541005 40506 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* W0811 05:32:07.736945 40506 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0811 05:32:07.737023 40506 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0811 05:32:07.737115 40506 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0811 05:32:07.737141 40506 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0811 05:32:07.737156 40506 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0811 05:32:07.737181 40506 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0811 05:32:07.737235 40506 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0811 05:32:07.738445 40506 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0811 05:32:07.741604 40506 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0811 05:32:07.746822 40506 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0811 05:32:07.749473 40506 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0811 05:32:07.749821 40506 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0811 05:32:07.778988 40506 target.go:112] Terraform output is in /tmp/test766009863/out --- PASS: TestMinimalGCE (0.24s) === RUN TestMinimalGCEPrivate I0811 05:32:07.783376 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:07.785205 40506 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* W0811 05:32:07.920490 40506 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0811 05:32:07.920553 40506 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0811 05:32:07.920642 40506 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/main" W0811 05:32:07.920666 40506 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/events" W0811 05:32:07.920681 40506 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W0811 05:32:07.920701 40506 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W0811 05:32:07.920743 40506 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0811 05:32:07.922286 40506 executor.go:111] Tasks: 0 done / 66 total; 40 can run I0811 05:32:07.925419 40506 executor.go:111] Tasks: 40 done / 66 total; 18 can run I0811 05:32:07.930614 40506 executor.go:111] Tasks: 58 done / 66 total; 4 can run I0811 05:32:07.932821 40506 executor.go:111] Tasks: 62 done / 66 total; 3 can run I0811 05:32:07.933513 40506 executor.go:111] Tasks: 65 done / 66 total; 1 can run I0811 05:32:07.933604 40506 executor.go:111] Tasks: 66 done / 66 total; 0 can run I0811 05:32:07.958622 40506 target.go:112] Terraform output is in /tmp/test3984530026/out --- PASS: TestMinimalGCEPrivate (0.18s) === RUN TestMinimalGCEInternalLoadBalancer I0811 05:32:07.962779 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:07.964758 40506 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* W0811 05:32:08.127538 40506 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0811 05:32:08.127618 40506 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0811 05:32:08.127747 40506 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/main" W0811 05:32:08.127785 40506 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/events" W0811 05:32:08.127813 40506 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W0811 05:32:08.127850 40506 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W0811 05:32:08.127918 40506 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0811 05:32:08.127965 40506 autoscalinggroup.go:284] Not hooking the instance group manager up to anything. I0811 05:32:08.129848 40506 executor.go:111] Tasks: 0 done / 68 total; 41 can run I0811 05:32:08.133297 40506 executor.go:111] Tasks: 41 done / 68 total; 17 can run W0811 05:32:08.138571 40506 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0811 05:32:08.138608 40506 executor.go:111] Tasks: 58 done / 68 total; 4 can run I0811 05:32:08.140933 40506 executor.go:111] Tasks: 62 done / 68 total; 3 can run I0811 05:32:08.142259 40506 executor.go:111] Tasks: 65 done / 68 total; 2 can run I0811 05:32:08.142654 40506 executor.go:111] Tasks: 67 done / 68 total; 1 can run I0811 05:32:08.142935 40506 executor.go:111] Tasks: 68 done / 68 total; 0 can run I0811 05:32:08.176564 40506 target.go:112] Terraform output is in /tmp/test3778966209/out --- PASS: TestMinimalGCEInternalLoadBalancer (0.22s) === RUN TestMinimalGCELongClusterName I0811 05:32:08.180769 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:08.183050 40506 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* W0811 05:32:08.372182 40506 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0811 05:32:08.372291 40506 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0811 05:32:08.372437 40506 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W0811 05:32:08.372476 40506 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W0811 05:32:08.372505 40506 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0811 05:32:08.372543 40506 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0811 05:32:08.372623 40506 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0811 05:32:08.374355 40506 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0811 05:32:08.377023 40506 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0811 05:32:08.382312 40506 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0811 05:32:08.386554 40506 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0811 05:32:08.386786 40506 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0811 05:32:08.421474 40506 target.go:112] Terraform output is in /tmp/test4196465306/out --- PASS: TestMinimalGCELongClusterName (0.25s) === RUN TestMinimalGCEInternalLoadBalancerLongClusterName I0811 05:32:08.428859 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:08.431894 40506 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* W0811 05:32:08.631824 40506 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0811 05:32:08.631920 40506 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0811 05:32:08.632065 40506 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W0811 05:32:08.632104 40506 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W0811 05:32:08.632135 40506 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0811 05:32:08.632172 40506 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0811 05:32:08.632242 40506 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0811 05:32:08.632294 40506 autoscalinggroup.go:284] Not hooking the instance group manager up to anything. I0811 05:32:08.634101 40506 executor.go:111] Tasks: 0 done / 67 total; 42 can run I0811 05:32:08.636680 40506 executor.go:111] Tasks: 42 done / 67 total; 16 can run W0811 05:32:08.641971 40506 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0811 05:32:08.642010 40506 executor.go:111] Tasks: 58 done / 67 total; 5 can run I0811 05:32:08.646257 40506 executor.go:111] Tasks: 63 done / 67 total; 2 can run I0811 05:32:08.646402 40506 executor.go:111] Tasks: 65 done / 67 total; 1 can run I0811 05:32:08.646506 40506 executor.go:111] Tasks: 66 done / 67 total; 1 can run I0811 05:32:08.646593 40506 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0811 05:32:08.667070 40506 target.go:112] Terraform output is in /tmp/test1003872663/out --- PASS: TestMinimalGCEInternalLoadBalancerLongClusterName (0.24s) === RUN TestHA I0811 05:32:08.669569 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:08.669723 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:08.669761 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:08.669794 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:08.669827 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:08.669881 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:08.669899 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:08.669921 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:08.669940 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:08.688921 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:08.690770 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:08.690831 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:08.690867 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:08.690905 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:08.726143 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:08.814459 40506 executor.go:111] Tasks: 0 done / 103 total; 49 can run W0811 05:32:08.814951 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:08.815057 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0811 05:32:08.815186 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:08.815099 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0811 05:32:08.815218 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:08.815234 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:08.815161 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.ha.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:08.815380 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:08.816692 40506 executor.go:111] Tasks: 49 done / 103 total; 23 can run I0811 05:32:08.825066 40506 executor.go:111] Tasks: 72 done / 103 total; 27 can run I0811 05:32:08.825330 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:08.825469 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:08.825578 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:08.825663 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:08.826569 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:08.826934 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:08.827018 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:08.828400 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:08.828526 40506 executor.go:111] Tasks: 99 done / 103 total; 4 can run I0811 05:32:08.828919 40506 executor.go:111] Tasks: 103 done / 103 total; 0 can run I0811 05:32:08.900623 40506 target.go:112] Terraform output is in /tmp/test1079001262/out --- PASS: TestHA (0.23s) === RUN TestHighAvailabilityGCE I0811 05:32:08.903217 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:08.904435 40506 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* W0811 05:32:09.029818 40506 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0811 05:32:09.029886 40506 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0811 05:32:09.030020 40506 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/main" W0811 05:32:09.030056 40506 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/events" W0811 05:32:09.030080 40506 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W0811 05:32:09.030101 40506 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W0811 05:32:09.030141 40506 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0811 05:32:09.030190 40506 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0811 05:32:09.030230 40506 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0811 05:32:09.031553 40506 executor.go:111] Tasks: 0 done / 79 total; 44 can run I0811 05:32:09.033596 40506 executor.go:111] Tasks: 44 done / 79 total; 20 can run I0811 05:32:09.039875 40506 executor.go:111] Tasks: 64 done / 79 total; 7 can run I0811 05:32:09.041459 40506 executor.go:111] Tasks: 71 done / 79 total; 5 can run I0811 05:32:09.042253 40506 executor.go:111] Tasks: 76 done / 79 total; 3 can run I0811 05:32:09.042430 40506 executor.go:111] Tasks: 79 done / 79 total; 0 can run I0811 05:32:09.067339 40506 target.go:112] Terraform output is in /tmp/test2583265742/out --- PASS: TestHighAvailabilityGCE (0.17s) === RUN TestComplex I0811 05:32:09.070279 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:09.070428 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:09.070468 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:09.070546 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:09.070579 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:09.070609 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:09.070651 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:09.070676 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:09.070693 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:09.091537 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0811 05:32:09.093824 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:09.093892 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:09.131238 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:09.277500 40506 executor.go:111] Tasks: 0 done / 120 total; 51 can run W0811 05:32:09.278579 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:09.278697 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:09.278868 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:09.278888 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:09.278976 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:09.284149 40506 executor.go:111] Tasks: 51 done / 120 total; 27 can run I0811 05:32:09.289938 40506 executor.go:111] Tasks: 78 done / 120 total; 39 can run I0811 05:32:09.290311 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:09.290577 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:09.292143 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:09.295044 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:09.295133 40506 executor.go:111] Tasks: 117 done / 120 total; 3 can run I0811 05:32:09.296428 40506 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0811 05:32:09.356041 40506 target.go:112] Terraform output is in /tmp/test2534298938/out I0811 05:32:09.358504 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:09.358630 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:09.358667 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:09.358698 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:09.358729 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:09.358755 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:09.358773 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:09.358797 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:09.358814 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:09.376701 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0811 05:32:09.378949 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:09.379009 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:09.414296 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0811 05:32:09.559524 40506 executor.go:111] Tasks: 0 done / 120 total; 51 can run W0811 05:32:09.559733 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:09.562163 40506 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0811 05:32:09.562202 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:09.562338 40506 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0811 05:32:09.563353 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:09.566569 40506 executor.go:111] Tasks: 51 done / 120 total; 27 can run I0811 05:32:09.572478 40506 executor.go:111] Tasks: 78 done / 120 total; 39 can run I0811 05:32:09.572901 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:09.573157 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:09.574034 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:09.576292 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:09.576431 40506 executor.go:111] Tasks: 117 done / 120 total; 3 can run I0811 05:32:09.576775 40506 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0811 05:32:09.581648 40506 target.go:168] Cloudformation output is in /tmp/test2633720833/out I0811 05:32:09.593233 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:09.593408 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:09.593448 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:09.593491 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:09.593550 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:09.593609 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:09.593646 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:09.593692 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:09.593728 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:09.615982 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0811 05:32:09.618757 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:09.618831 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:09.654626 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:09.838305 40506 executor.go:111] Tasks: 0 done / 120 total; 51 can run I0811 05:32:09.838695 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:09.838741 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:09.838889 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0811 05:32:09.839058 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:09.841794 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:09.845334 40506 executor.go:111] Tasks: 51 done / 120 total; 27 can run I0811 05:32:09.851265 40506 executor.go:111] Tasks: 78 done / 120 total; 39 can run I0811 05:32:09.851697 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:09.852045 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:09.853077 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:09.853884 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:09.853959 40506 executor.go:111] Tasks: 117 done / 120 total; 3 can run I0811 05:32:09.854405 40506 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0811 05:32:09.920497 40506 target.go:112] Terraform output is in /tmp/test3022778698/out --- PASS: TestComplex (0.86s) === RUN TestCompress I0811 05:32:09.925533 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:09.925705 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:09.925747 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:09.925782 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:09.925820 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:09.925877 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:09.925894 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:09.925916 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:09.925936 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:09.940099 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:09.941753 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:09.941815 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:09.976945 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:10.056176 40506 executor.go:111] Tasks: 0 done / 84 total; 42 can run W0811 05:32:10.056711 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:10.056821 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:10.056858 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0811 05:32:10.056949 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:10.056956 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:10.058724 40506 executor.go:111] Tasks: 42 done / 84 total; 19 can run I0811 05:32:10.065453 40506 executor.go:111] Tasks: 61 done / 84 total; 21 can run I0811 05:32:10.065672 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:10.065812 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:10.067242 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:10.068361 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:10.068428 40506 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0811 05:32:10.068651 40506 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0811 05:32:10.110931 40506 target.go:112] Terraform output is in /tmp/test3001081266/out --- PASS: TestCompress (0.19s) === RUN TestExternalPolicies I0811 05:32:10.113200 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:10.113349 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:10.113387 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:10.113418 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:10.113450 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:10.113511 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:10.113529 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:10.113551 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:10.113571 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:10.137927 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:10.141328 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:10.141438 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:10.177534 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:10.259797 40506 executor.go:111] Tasks: 0 done / 98 total; 45 can run W0811 05:32:10.260265 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:10.260362 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:10.260367 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externalpolicies.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:10.261510 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:10.261560 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:10.261684 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:10.267299 40506 executor.go:111] Tasks: 45 done / 98 total; 20 can run I0811 05:32:10.273255 40506 executor.go:111] Tasks: 65 done / 98 total; 30 can run I0811 05:32:10.273570 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:10.273855 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:10.274710 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:10.275242 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:10.275314 40506 executor.go:111] Tasks: 95 done / 98 total; 3 can run I0811 05:32:10.275538 40506 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0811 05:32:10.329657 40506 target.go:112] Terraform output is in /tmp/test3776127563/out --- PASS: TestExternalPolicies (0.22s) === RUN TestMinimalIPv6 I0811 05:32:10.333390 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:10.333597 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:10.333648 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:10.333698 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:10.333748 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:10.333838 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:10.333871 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:10.333912 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:10.333942 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:10.361946 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:10.363918 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:10.363983 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:10.399760 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:10.567705 40506 executor.go:111] Tasks: 0 done / 104 total; 45 can run W0811 05:32:10.567975 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:10.568438 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0811 05:32:10.568469 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:10.568484 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:10.568647 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:10.568752 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:10.571010 40506 executor.go:111] Tasks: 45 done / 104 total; 21 can run I0811 05:32:10.579423 40506 executor.go:111] Tasks: 66 done / 104 total; 30 can run I0811 05:32:10.579815 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:10.579957 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:10.580959 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:10.581489 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:10.581568 40506 executor.go:111] Tasks: 96 done / 104 total; 5 can run I0811 05:32:10.581962 40506 executor.go:111] Tasks: 101 done / 104 total; 3 can run I0811 05:32:10.582341 40506 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0811 05:32:10.642606 40506 target.go:112] Terraform output is in /tmp/test204702474/out I0811 05:32:10.646781 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:10.647000 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:10.647062 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:10.647113 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:10.647169 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:10.647212 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:10.647241 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:10.647281 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:10.647309 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:10.675533 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:10.678960 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:10.679104 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:10.715073 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0811 05:32:10.861257 40506 executor.go:111] Tasks: 0 done / 104 total; 45 can run I0811 05:32:10.861912 40506 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0811 05:32:10.861964 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:10.862050 40506 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0811 05:32:10.862317 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:10.862342 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:10.862405 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0811 05:32:10.862539 40506 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0811 05:32:10.862567 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:10.862691 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0811 05:32:10.864937 40506 executor.go:111] Tasks: 45 done / 104 total; 21 can run I0811 05:32:10.871423 40506 executor.go:111] Tasks: 66 done / 104 total; 30 can run I0811 05:32:10.871857 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:10.872144 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:10.873823 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:10.877405 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:10.878390 40506 executor.go:111] Tasks: 96 done / 104 total; 5 can run I0811 05:32:10.878911 40506 executor.go:111] Tasks: 101 done / 104 total; 3 can run I0811 05:32:10.879229 40506 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0811 05:32:10.883163 40506 target.go:168] Cloudformation output is in /tmp/test3131703626/out --- PASS: TestMinimalIPv6 (0.56s) === RUN TestMinimalIPv6Private I0811 05:32:10.894636 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:10.894861 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:10.894912 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:10.894959 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:10.895007 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:10.895086 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:10.895117 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:10.895156 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:10.895185 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:10.924437 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* I0811 05:32:10.927673 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:10.927752 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:10.964815 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:11.143153 40506 executor.go:111] Tasks: 0 done / 122 total; 46 can run I0811 05:32:11.143785 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0811 05:32:11.143924 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:11.143830 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:11.143989 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0811 05:32:11.144148 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:11.144149 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:11.150785 40506 executor.go:111] Tasks: 46 done / 122 total; 23 can run I0811 05:32:11.156877 40506 executor.go:111] Tasks: 69 done / 122 total; 36 can run I0811 05:32:11.157495 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:11.157719 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:11.158924 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:11.162037 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:11.162176 40506 executor.go:111] Tasks: 105 done / 122 total; 11 can run I0811 05:32:11.162782 40506 executor.go:111] Tasks: 116 done / 122 total; 6 can run I0811 05:32:11.163186 40506 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0811 05:32:11.237495 40506 target.go:112] Terraform output is in /tmp/test1393664931/out --- PASS: TestMinimalIPv6Private (0.35s) === RUN TestMinimalIPv6Calico I0811 05:32:11.242176 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:11.242454 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:11.242517 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:11.242572 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:11.242628 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:11.242721 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:11.242753 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:11.242790 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:11.242824 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:11.272594 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0811 05:32:11.276243 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:11.276352 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:11.313079 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:11.731580 40506 executor.go:111] Tasks: 0 done / 107 total; 47 can run W0811 05:32:11.731947 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:11.731978 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:11.732013 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:11.732105 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:11.732104 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:11.732435 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:11.739565 40506 executor.go:111] Tasks: 47 done / 107 total; 21 can run I0811 05:32:11.745350 40506 executor.go:111] Tasks: 68 done / 107 total; 31 can run I0811 05:32:11.745819 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:11.746024 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:11.747076 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:11.748895 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:11.749052 40506 executor.go:111] Tasks: 99 done / 107 total; 5 can run I0811 05:32:11.749541 40506 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0811 05:32:11.749921 40506 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0811 05:32:11.815388 40506 target.go:112] Terraform output is in /tmp/test321614280/out I0811 05:32:11.821437 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:11.821694 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:11.821765 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:11.821831 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:11.821902 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:11.821956 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:11.822002 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:11.822054 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:11.822098 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:11.848687 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0811 05:32:11.852256 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:11.852416 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:11.888858 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0811 05:32:12.321302 40506 executor.go:111] Tasks: 0 done / 107 total; 47 can run I0811 05:32:12.321583 40506 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0811 05:32:12.321617 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:12.321712 40506 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0811 05:32:12.321796 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:12.321914 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:12.322042 40506 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key W0811 05:32:12.322124 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:12.322058 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:12.322181 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0811 05:32:12.325209 40506 executor.go:111] Tasks: 47 done / 107 total; 21 can run I0811 05:32:12.331408 40506 executor.go:111] Tasks: 68 done / 107 total; 31 can run I0811 05:32:12.331887 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:12.332165 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:12.333335 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:12.335440 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:12.335662 40506 executor.go:111] Tasks: 99 done / 107 total; 5 can run I0811 05:32:12.336362 40506 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0811 05:32:12.336800 40506 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0811 05:32:12.342740 40506 target.go:168] Cloudformation output is in /tmp/test1410284997/out --- PASS: TestMinimalIPv6Calico (1.11s) === RUN TestMinimalIPv6Cilium I0811 05:32:12.354614 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:12.354851 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:12.354908 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:12.354959 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:12.355013 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:12.355094 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:12.355123 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:12.355163 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:12.355192 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:12.383881 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:12.387790 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:12.387878 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:12.425013 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:12.643926 40506 executor.go:111] Tasks: 0 done / 105 total; 46 can run W0811 05:32:12.644543 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:12.644586 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:12.644804 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:12.644788 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:12.644838 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:12.644963 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:12.646857 40506 executor.go:111] Tasks: 46 done / 105 total; 21 can run I0811 05:32:12.655512 40506 executor.go:111] Tasks: 67 done / 105 total; 30 can run I0811 05:32:12.656054 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:12.656279 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:12.657527 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:12.658919 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:12.659146 40506 executor.go:111] Tasks: 97 done / 105 total; 5 can run I0811 05:32:12.659543 40506 executor.go:111] Tasks: 102 done / 105 total; 3 can run I0811 05:32:12.659907 40506 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0811 05:32:12.722577 40506 target.go:112] Terraform output is in /tmp/test2876547032/out I0811 05:32:12.726949 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:12.727167 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:12.727222 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:12.727275 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:12.727329 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:12.727371 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:12.727399 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:12.727437 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:12.727467 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:12.752725 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:12.756233 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:12.756327 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:12.793198 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0811 05:32:13.011665 40506 executor.go:111] Tasks: 0 done / 105 total; 46 can run I0811 05:32:13.012056 40506 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0811 05:32:13.012107 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:13.012218 40506 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0811 05:32:13.012381 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:13.012809 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:13.012895 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0811 05:32:13.013037 40506 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0811 05:32:13.013072 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:13.013226 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0811 05:32:13.016507 40506 executor.go:111] Tasks: 46 done / 105 total; 21 can run I0811 05:32:13.024811 40506 executor.go:111] Tasks: 67 done / 105 total; 30 can run I0811 05:32:13.025459 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:13.025709 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:13.026646 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:13.028215 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:13.028373 40506 executor.go:111] Tasks: 97 done / 105 total; 5 can run I0811 05:32:13.028839 40506 executor.go:111] Tasks: 102 done / 105 total; 3 can run I0811 05:32:13.029160 40506 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0811 05:32:13.033141 40506 target.go:168] Cloudformation output is in /tmp/test990590386/out --- PASS: TestMinimalIPv6Cilium (0.69s) === RUN TestMinimalWarmPool I0811 05:32:13.044994 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:13.045225 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:13.045277 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:13.045323 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:13.045373 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:13.045453 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:13.045483 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:13.045521 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:13.045549 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:13.074264 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:13.077665 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:13.077770 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:13.113681 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:13.315747 40506 executor.go:111] Tasks: 0 done / 88 total; 45 can run W0811 05:32:13.316150 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:13.316376 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-warmpool.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:13.318436 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:13.318828 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:13.318871 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:13.319010 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:13.322834 40506 executor.go:111] Tasks: 45 done / 88 total; 19 can run I0811 05:32:13.328471 40506 executor.go:111] Tasks: 64 done / 88 total; 21 can run I0811 05:32:13.328970 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:13.329212 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:13.330233 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:13.331825 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:13.331951 40506 executor.go:111] Tasks: 85 done / 88 total; 2 can run I0811 05:32:13.332257 40506 executor.go:111] Tasks: 87 done / 88 total; 1 can run I0811 05:32:13.332637 40506 executor.go:111] Tasks: 88 done / 88 total; 0 can run I0811 05:32:13.384992 40506 target.go:112] Terraform output is in /tmp/test4090958151/out --- PASS: TestMinimalWarmPool (0.34s) === RUN TestMinimalEtcd I0811 05:32:13.389554 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:13.389805 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:13.389867 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:13.389977 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:13.390032 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:13.390078 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:13.390110 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:13.390149 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:13.390180 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:13.419609 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:13.423037 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:13.423151 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:13.460121 40506 zones.go:93] ListHostedZonesPages { } W0811 05:32:13.548004 40506 model.go:247] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W0811 05:32:13.548061 40506 model.go:347] overriding log level in manifest (embedded etcd manifest), new level is 10 W0811 05:32:13.549921 40506 model.go:247] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W0811 05:32:13.550088 40506 model.go:484] overloading ENV var in manifest (embedded etcd manifest) with ETCD_MANAGER_HOURLY_BACKUPS_RETENTION=1d W0811 05:32:13.550140 40506 model.go:484] overloading ENV var in manifest (embedded etcd manifest) with ETCD_MANAGER_DAILY_BACKUPS_RETENTION=30d ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0811 05:32:13.562542 40506 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0811 05:32:13.563255 40506 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0811 05:32:13.563289 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:13.563381 40506 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0811 05:32:13.563401 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:13.563371 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:13.563509 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0811 05:32:13.563560 40506 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0811 05:32:13.563595 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:13.563772 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-etcd.example.com" },{ Key: "KubernetesCluster", Value: "minimal-etcd.example.com" },{ Key: "kubernetes.io/cluster/minimal-etcd.example.com", Value: "owned" }] }] } I0811 05:32:13.565370 40506 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0811 05:32:13.574105 40506 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0811 05:32:13.574480 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:13.574703 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:13.575673 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:13.577324 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:13.577458 40506 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0811 05:32:13.577732 40506 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0811 05:32:13.580744 40506 target.go:168] Cloudformation output is in /tmp/test183408798/out --- PASS: TestMinimalEtcd (0.20s) === RUN TestMinimalGp3 I0811 05:32:13.591789 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:13.592007 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:13.592058 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:13.592107 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:13.592158 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:13.592236 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:13.592266 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:13.592324 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:13.592382 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:13.616369 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:13.618833 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:13.618914 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:13.654959 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:13.757248 40506 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0811 05:32:13.758148 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0811 05:32:13.758224 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:13.758199 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0811 05:32:13.758335 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:13.758339 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:13.758369 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:13.764337 40506 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0811 05:32:13.769932 40506 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0811 05:32:13.770156 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:13.770534 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:13.771695 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:13.772712 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:13.772834 40506 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0811 05:32:13.773190 40506 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0811 05:32:13.826099 40506 target.go:112] Terraform output is in /tmp/test1099166543/out I0811 05:32:13.829912 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:13.830133 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:13.830189 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:13.830241 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:13.830295 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:13.830339 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:13.830368 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:13.830405 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:13.830435 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:13.855342 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:13.858236 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:13.858335 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:13.897457 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0811 05:32:13.995920 40506 executor.go:111] Tasks: 0 done / 85 total; 43 can run W0811 05:32:13.996424 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:13.996422 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:13.996558 40506 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0811 05:32:13.996580 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:13.996733 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } W0811 05:32:13.996929 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:13.997028 40506 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0811 05:32:13.997070 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:13.997189 40506 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0811 05:32:13.999040 40506 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0811 05:32:14.008357 40506 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0811 05:32:14.008651 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.009083 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.010151 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.010995 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.011126 40506 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0811 05:32:14.011374 40506 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0811 05:32:14.014314 40506 target.go:168] Cloudformation output is in /tmp/test678353162/out --- PASS: TestMinimalGp3 (0.43s) === RUN TestMinimalLongClusterName I0811 05:32:14.024608 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:14.024832 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:14.024896 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:14.024944 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:14.024998 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:14.025076 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:14.025106 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:14.025142 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:14.025171 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:14.048848 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:14.051659 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.051747 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0811 05:32:14.089163 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:14.190568 40506 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0811 05:32:14.190913 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:14.191466 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:14.191746 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:14.193465 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:14.193632 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0811 05:32:14.193967 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:14.198529 40506 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0811 05:32:14.204069 40506 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0811 05:32:14.204563 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.204789 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.206067 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.207021 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.207147 40506 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0811 05:32:14.207516 40506 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0811 05:32:14.266672 40506 target.go:112] Terraform output is in /tmp/test102622471/out I0811 05:32:14.271269 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:14.271602 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:14.271703 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:14.271784 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:14.271841 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:14.271885 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:14.271916 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:14.271953 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:14.271984 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:14.295890 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:14.299291 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.299393 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0811 05:32:14.336289 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0811 05:32:14.436974 40506 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0811 05:32:14.437300 40506 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0811 05:32:14.437335 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:14.437391 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:14.437586 40506 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0811 05:32:14.437618 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:14.437768 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "this.is.truly.a.really.really.long.cluster-name.minimal.example.com" },{ Key: "KubernetesCluster", Value: "this.is.truly.a.really.really.long.cluster-name.minimal.example.com" },{ Key: "kubernetes.io/cluster/this.is.truly.a.really.really.long.cluster-name.minimal.example.com", Value: "owned" }] }] } W0811 05:32:14.437949 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:14.437439 40506 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0811 05:32:14.438072 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:14.443701 40506 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0811 05:32:14.449196 40506 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0811 05:32:14.449475 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.449670 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.450699 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.451758 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.451892 40506 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0811 05:32:14.452138 40506 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0811 05:32:14.455676 40506 target.go:168] Cloudformation output is in /tmp/test3327550205/out --- PASS: TestMinimalLongClusterName (0.44s) === RUN TestExistingIAMCloudformation I0811 05:32:14.466498 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:14.466721 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:14.466776 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:14.466865 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:14.466919 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:14.466960 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:14.466989 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:14.467028 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:14.467054 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:14.494718 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:14.497614 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.497715 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.534192 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0811 05:32:14.636489 40506 executor.go:111] Tasks: 0 done / 75 total; 41 can run W0811 05:32:14.636731 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:14.636784 40506 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0811 05:32:14.636826 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:14.636935 40506 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0811 05:32:14.636991 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:14.637218 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:14.637358 40506 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0811 05:32:14.637374 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:14.637473 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0811 05:32:14.640023 40506 executor.go:111] Tasks: 41 done / 75 total; 11 can run I0811 05:32:14.645089 40506 executor.go:111] Tasks: 52 done / 75 total; 21 can run I0811 05:32:14.645527 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.645785 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.646643 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.648023 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.648156 40506 executor.go:111] Tasks: 73 done / 75 total; 2 can run I0811 05:32:14.648429 40506 executor.go:111] Tasks: 75 done / 75 total; 0 can run I0811 05:32:14.650636 40506 target.go:168] Cloudformation output is in /tmp/test2519036442/out --- PASS: TestExistingIAMCloudformation (0.19s) === RUN TestExistingSG I0811 05:32:14.658460 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:14.658674 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:14.658723 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:14.658766 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:14.658820 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:14.658897 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:14.658929 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:14.658967 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:14.658996 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:14.697900 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:14.701124 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.701225 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.701286 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.701344 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.739435 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:14.846376 40506 executor.go:111] Tasks: 0 done / 138 total; 49 can run I0811 05:32:14.846940 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existingsg.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:14.847055 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:14.847125 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:14.847505 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0811 05:32:14.847521 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:14.847524 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:14.847551 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:14.847684 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:14.849177 40506 executor.go:111] Tasks: 49 done / 138 total; 26 can run I0811 05:32:14.860367 40506 executor.go:111] Tasks: 75 done / 138 total; 58 can run I0811 05:32:14.860969 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.861163 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.861408 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.861794 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.862410 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.863709 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.864005 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.864092 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:14.864222 40506 executor.go:111] Tasks: 133 done / 138 total; 5 can run I0811 05:32:14.864695 40506 executor.go:111] Tasks: 138 done / 138 total; 0 can run I0811 05:32:14.959170 40506 target.go:112] Terraform output is in /tmp/test810300784/out --- PASS: TestExistingSG (0.31s) === RUN TestBastionAdditionalUserData I0811 05:32:14.963925 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:14.964179 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:14.964239 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:14.964291 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:14.964348 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:14.964448 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:14.964480 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:14.964519 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:14.964550 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:14.997548 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:15.001135 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:15.001245 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:15.001311 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:15.038491 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:15.143332 40506 executor.go:111] Tasks: 0 done / 119 total; 48 can run I0811 05:32:15.144003 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:15.144034 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0811 05:32:15.144111 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:15.144118 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0811 05:32:15.144183 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:15.144293 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.bastionuserdata.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:15.144411 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:15.146890 40506 executor.go:111] Tasks: 48 done / 119 total; 29 can run I0811 05:32:15.156251 40506 executor.go:111] Tasks: 77 done / 119 total; 36 can run I0811 05:32:15.156598 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:15.156983 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:15.157208 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:15.157829 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:15.158484 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:15.159341 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:15.159472 40506 executor.go:111] Tasks: 113 done / 119 total; 5 can run I0811 05:32:15.159997 40506 executor.go:111] Tasks: 118 done / 119 total; 1 can run I0811 05:32:15.160230 40506 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0811 05:32:15.238724 40506 target.go:112] Terraform output is in /tmp/test2959321728/out --- PASS: TestBastionAdditionalUserData (0.28s) === RUN TestPrivateWeave I0811 05:32:15.243344 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:15.243594 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:15.243653 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:15.243703 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:15.243758 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:15.243850 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:15.243884 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:15.243922 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:15.243953 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:15.271466 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:15.274582 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:15.274679 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:15.274742 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:15.486031 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:15.607765 40506 executor.go:111] Tasks: 0 done / 117 total; 47 can run I0811 05:32:15.608225 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0811 05:32:15.608315 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:15.608267 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:15.608404 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0811 05:32:15.608539 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:15.610075 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:15.611088 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateweave.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:15.615585 40506 executor.go:111] Tasks: 47 done / 117 total; 28 can run I0811 05:32:15.623315 40506 executor.go:111] Tasks: 75 done / 117 total; 36 can run I0811 05:32:15.623741 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:15.623888 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:15.624090 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:15.624267 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:15.624813 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:15.626419 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:15.626547 40506 executor.go:111] Tasks: 111 done / 117 total; 5 can run I0811 05:32:15.627112 40506 executor.go:111] Tasks: 116 done / 117 total; 1 can run I0811 05:32:15.627269 40506 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0811 05:32:15.709073 40506 target.go:112] Terraform output is in /tmp/test951853569/out --- PASS: TestPrivateWeave (0.47s) === RUN TestPrivateFlannel I0811 05:32:15.713649 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:15.713900 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:15.713963 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:15.714017 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:15.714073 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:15.714169 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:15.714200 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:15.714238 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:15.714269 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:15.746674 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:15.750463 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:15.750675 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:15.750840 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:15.787429 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:15.911168 40506 executor.go:111] Tasks: 0 done / 117 total; 47 can run W0811 05:32:15.911564 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:15.911716 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:15.911923 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:15.911969 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:15.912114 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:15.912155 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateflannel.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:15.912254 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:15.914542 40506 executor.go:111] Tasks: 47 done / 117 total; 28 can run I0811 05:32:15.919904 40506 executor.go:111] Tasks: 75 done / 117 total; 36 can run I0811 05:32:15.920090 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:15.920362 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:15.922378 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:15.922497 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:15.923128 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:15.927996 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:15.928150 40506 executor.go:111] Tasks: 111 done / 117 total; 5 can run I0811 05:32:15.928761 40506 executor.go:111] Tasks: 116 done / 117 total; 1 can run I0811 05:32:15.928949 40506 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0811 05:32:16.004535 40506 target.go:112] Terraform output is in /tmp/test3960266418/out --- PASS: TestPrivateFlannel (0.30s) === RUN TestPrivateCalico I0811 05:32:16.009075 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:16.009331 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:16.009394 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:16.009452 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:16.009506 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:16.009620 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:16.009650 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:16.009688 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:16.009719 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:16.042111 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0811 05:32:16.045939 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:16.046045 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:16.046488 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:16.085043 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:16.497830 40506 executor.go:111] Tasks: 0 done / 124 total; 50 can run I0811 05:32:16.499112 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:16.499331 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:16.499402 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:16.499583 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:16.501101 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:16.501149 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:16.501290 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:16.505624 40506 executor.go:111] Tasks: 50 done / 124 total; 28 can run I0811 05:32:16.511606 40506 executor.go:111] Tasks: 78 done / 124 total; 40 can run I0811 05:32:16.511967 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:16.512154 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:16.512389 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:16.512634 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:16.513866 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:16.517712 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:16.517847 40506 executor.go:111] Tasks: 118 done / 124 total; 5 can run I0811 05:32:16.518302 40506 executor.go:111] Tasks: 123 done / 124 total; 1 can run I0811 05:32:16.518468 40506 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0811 05:32:16.592593 40506 target.go:112] Terraform output is in /tmp/test4180370381/out I0811 05:32:16.598253 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:16.598479 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:16.598542 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:16.598593 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:16.598647 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:16.598692 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:16.598721 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:16.598759 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:16.598791 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:16.632761 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0811 05:32:16.636571 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:16.636671 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:16.636738 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:16.673974 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0811 05:32:17.087958 40506 executor.go:111] Tasks: 0 done / 124 total; 50 can run W0811 05:32:17.088311 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:17.088422 40506 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0811 05:32:17.088472 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:17.088457 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:17.088594 40506 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0811 05:32:17.088706 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:17.088860 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:17.090080 40506 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0811 05:32:17.090110 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:17.090287 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "privatecalico.example.com" }] }] } I0811 05:32:17.096531 40506 executor.go:111] Tasks: 50 done / 124 total; 28 can run I0811 05:32:17.102455 40506 executor.go:111] Tasks: 78 done / 124 total; 40 can run I0811 05:32:17.103163 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.103367 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.103453 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.103695 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.104884 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.105894 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.106027 40506 executor.go:111] Tasks: 118 done / 124 total; 5 can run I0811 05:32:17.106452 40506 executor.go:111] Tasks: 123 done / 124 total; 1 can run I0811 05:32:17.106605 40506 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0811 05:32:17.110933 40506 target.go:168] Cloudformation output is in /tmp/test608161838/out --- PASS: TestPrivateCalico (1.12s) === RUN TestPrivateCilium I0811 05:32:17.127442 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:17.127685 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:17.127741 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:17.127788 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:17.127842 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:17.127929 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:17.127960 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:17.127996 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:17.128026 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:17.156470 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:17.159650 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.159742 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.159805 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.196482 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:17.349297 40506 executor.go:111] Tasks: 0 done / 117 total; 47 can run W0811 05:32:17.349823 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:17.349949 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:17.350184 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0811 05:32:17.350220 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:17.350240 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:17.350339 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0811 05:32:17.350369 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:17.353713 40506 executor.go:111] Tasks: 47 done / 117 total; 28 can run I0811 05:32:17.362168 40506 executor.go:111] Tasks: 75 done / 117 total; 36 can run I0811 05:32:17.362891 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.363102 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.363295 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.363378 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.364004 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.366183 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.366312 40506 executor.go:111] Tasks: 111 done / 117 total; 5 can run I0811 05:32:17.366720 40506 executor.go:111] Tasks: 116 done / 117 total; 1 can run I0811 05:32:17.366892 40506 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0811 05:32:17.444355 40506 target.go:112] Terraform output is in /tmp/test3674254007/out I0811 05:32:17.448722 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:17.448953 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:17.449043 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:17.449105 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:17.449162 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:17.449211 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:17.449241 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:17.449282 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:17.449313 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:17.477297 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:17.480461 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.480553 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.480613 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.517565 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0811 05:32:17.666440 40506 executor.go:111] Tasks: 0 done / 117 total; 47 can run W0811 05:32:17.666809 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0811 05:32:17.667175 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:17.667205 40506 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0811 05:32:17.667245 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:17.667341 40506 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0811 05:32:17.667521 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:17.667544 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:17.667781 40506 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0811 05:32:17.667812 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:17.667954 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/privatecilium.example.com", Value: "owned" },{ Key: "Name", Value: "privatecilium.example.com" },{ Key: "KubernetesCluster", Value: "privatecilium.example.com" }] }] } I0811 05:32:17.669426 40506 executor.go:111] Tasks: 47 done / 117 total; 28 can run I0811 05:32:17.678746 40506 executor.go:111] Tasks: 75 done / 117 total; 36 can run I0811 05:32:17.679193 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.679439 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.679523 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.679800 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.680643 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.682150 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.682293 40506 executor.go:111] Tasks: 111 done / 117 total; 5 can run I0811 05:32:17.682772 40506 executor.go:111] Tasks: 116 done / 117 total; 1 can run I0811 05:32:17.683104 40506 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0811 05:32:17.688059 40506 target.go:168] Cloudformation output is in /tmp/test2983319236/out --- PASS: TestPrivateCilium (0.57s) === RUN TestPrivateCilium2 I0811 05:32:17.702372 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:17.702610 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:17.702659 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:17.702706 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:17.702757 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:17.702839 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:17.702873 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:17.702910 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:17.702939 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:17.734746 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.19.16 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.19.16 ********************************************************************************* I0811 05:32:17.738254 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.738354 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.738418 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.776857 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:17.919514 40506 executor.go:111] Tasks: 0 done / 117 total; 47 can run I0811 05:32:17.919833 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:17.919878 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:17.920012 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0811 05:32:17.920048 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:17.920509 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:17.920587 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:17.920650 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:17.928268 40506 executor.go:111] Tasks: 47 done / 117 total; 28 can run I0811 05:32:17.934278 40506 executor.go:111] Tasks: 75 done / 117 total; 36 can run I0811 05:32:17.934755 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.934899 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.935064 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.935302 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.935886 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.937289 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:17.937415 40506 executor.go:111] Tasks: 111 done / 117 total; 5 can run I0811 05:32:17.937861 40506 executor.go:111] Tasks: 116 done / 117 total; 1 can run I0811 05:32:17.938051 40506 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0811 05:32:18.015906 40506 target.go:112] Terraform output is in /tmp/test69449172/out I0811 05:32:18.020368 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:18.020598 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:18.020660 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:18.020717 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:18.020775 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:18.020824 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:18.020854 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:18.020895 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:18.020924 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:18.049339 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.19.16 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.19.16 ********************************************************************************* I0811 05:32:18.052768 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.052866 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.052935 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.089800 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0811 05:32:18.229126 40506 executor.go:111] Tasks: 0 done / 117 total; 47 can run W0811 05:32:18.229459 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:18.229722 40506 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0811 05:32:18.229722 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:18.229753 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0811 05:32:18.229852 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0811 05:32:18.229831 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:18.229872 40506 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0811 05:32:18.229916 40506 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0811 05:32:18.229946 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:18.230101 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "privatecilium.example.com" },{ Key: "kubernetes.io/cluster/privatecilium.example.com", Value: "owned" },{ Key: "Name", Value: "privatecilium.example.com" }] }] } I0811 05:32:18.232541 40506 executor.go:111] Tasks: 47 done / 117 total; 28 can run I0811 05:32:18.237757 40506 executor.go:111] Tasks: 75 done / 117 total; 36 can run I0811 05:32:18.238001 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.238266 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.238406 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.238573 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.239106 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.240774 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.240921 40506 executor.go:111] Tasks: 111 done / 117 total; 5 can run I0811 05:32:18.242004 40506 executor.go:111] Tasks: 116 done / 117 total; 1 can run I0811 05:32:18.242534 40506 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0811 05:32:18.249169 40506 target.go:168] Cloudformation output is in /tmp/test1885052206/out --- PASS: TestPrivateCilium2 (0.56s) === RUN TestPrivateCiliumAdvanced I0811 05:32:18.262452 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:18.262697 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:18.262750 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:18.262799 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:18.262857 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:18.262939 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:18.262970 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:18.263009 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:18.263038 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:18.295388 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:18.299249 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.299341 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.299401 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.335943 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:18.492822 40506 executor.go:111] Tasks: 0 done / 123 total; 53 can run W0811 05:32:18.493294 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:18.493757 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:18.493813 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:18.493803 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:18.493885 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0811 05:32:18.493994 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:18.494042 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:18.497757 40506 executor.go:111] Tasks: 53 done / 123 total; 28 can run I0811 05:32:18.507584 40506 executor.go:111] Tasks: 81 done / 123 total; 36 can run I0811 05:32:18.508214 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.508406 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.508498 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.508666 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.509633 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.511031 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.511158 40506 executor.go:111] Tasks: 117 done / 123 total; 5 can run I0811 05:32:18.511645 40506 executor.go:111] Tasks: 122 done / 123 total; 1 can run I0811 05:32:18.511834 40506 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0811 05:32:18.591352 40506 target.go:112] Terraform output is in /tmp/test4266291620/out I0811 05:32:18.596026 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:18.596251 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:18.596314 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:18.596370 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:18.596426 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:18.596474 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:18.596506 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:18.596545 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:18.596575 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:18.628866 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:18.631853 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.631937 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.631993 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.668667 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0811 05:32:18.824673 40506 executor.go:111] Tasks: 0 done / 123 total; 53 can run I0811 05:32:18.825140 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:18.825191 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0811 05:32:18.825202 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0811 05:32:18.825272 40506 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0811 05:32:18.825290 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:18.825395 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privateciliumadvanced.example.com" },{ Key: "KubernetesCluster", Value: "privateciliumadvanced.example.com" },{ Key: "kubernetes.io/cluster/privateciliumadvanced.example.com", Value: "owned" }] }] } I0811 05:32:18.825516 40506 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0811 05:32:18.825546 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0811 05:32:18.825634 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:18.825650 40506 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0811 05:32:18.833113 40506 executor.go:111] Tasks: 53 done / 123 total; 28 can run I0811 05:32:18.839362 40506 executor.go:111] Tasks: 81 done / 123 total; 36 can run I0811 05:32:18.839703 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.839921 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.840129 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.840251 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.841259 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.842503 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.842644 40506 executor.go:111] Tasks: 117 done / 123 total; 5 can run I0811 05:32:18.843094 40506 executor.go:111] Tasks: 122 done / 123 total; 1 can run I0811 05:32:18.843246 40506 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0811 05:32:18.847650 40506 target.go:168] Cloudformation output is in /tmp/test919165566/out --- PASS: TestPrivateCiliumAdvanced (0.60s) === RUN TestPrivateCanal I0811 05:32:18.864577 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:18.864819 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:18.864883 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:18.864972 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:18.865061 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:18.865209 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:18.865263 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:18.865333 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:18.865390 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:18.892700 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.10 ********************************************************************************* I0811 05:32:18.895533 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.895622 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.895714 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:18.934983 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:19.343606 40506 executor.go:111] Tasks: 0 done / 118 total; 48 can run I0811 05:32:19.344179 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecanal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:19.344214 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:19.344749 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:19.346557 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:19.346981 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:19.347029 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:19.347190 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:19.351789 40506 executor.go:111] Tasks: 48 done / 118 total; 28 can run I0811 05:32:19.357280 40506 executor.go:111] Tasks: 76 done / 118 total; 36 can run I0811 05:32:19.357631 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:19.357942 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:19.358226 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:19.358416 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:19.359203 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:19.360155 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:19.360279 40506 executor.go:111] Tasks: 112 done / 118 total; 5 can run I0811 05:32:19.360712 40506 executor.go:111] Tasks: 117 done / 118 total; 1 can run I0811 05:32:19.360917 40506 executor.go:111] Tasks: 118 done / 118 total; 0 can run I0811 05:32:19.438325 40506 target.go:112] Terraform output is in /tmp/test735993641/out --- PASS: TestPrivateCanal (0.58s) === RUN TestPrivateKopeio I0811 05:32:19.443701 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:19.443946 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:19.444002 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:19.444053 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:19.444110 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:19.444204 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:19.444235 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:19.444275 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:19.444306 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:19.473463 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:19.476374 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:19.476457 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:19.476515 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:19.513978 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:19.630270 40506 executor.go:111] Tasks: 0 done / 123 total; 47 can run W0811 05:32:19.630557 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:19.630754 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:19.630906 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:19.630984 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:19.631014 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:19.631105 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:19.631436 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:19.633861 40506 executor.go:111] Tasks: 47 done / 123 total; 31 can run I0811 05:32:19.639388 40506 executor.go:111] Tasks: 78 done / 123 total; 39 can run I0811 05:32:19.639786 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:19.639963 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:19.640218 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:19.640376 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:19.641176 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:19.641999 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:19.642131 40506 executor.go:111] Tasks: 117 done / 123 total; 6 can run I0811 05:32:19.642558 40506 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0811 05:32:19.725299 40506 target.go:112] Terraform output is in /tmp/test386636137/out --- PASS: TestPrivateKopeio (0.29s) === RUN TestUnmanaged I0811 05:32:19.729765 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:19.730011 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:19.730072 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:19.730124 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:19.730179 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:19.730272 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:19.730304 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:19.730345 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:19.730376 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:19.759686 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:19.762592 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:19.762673 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:19.762728 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:19.813114 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:19.912939 40506 executor.go:111] Tasks: 0 done / 105 total; 45 can run W0811 05:32:19.913159 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:19.913616 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.unmanaged.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:19.913853 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0811 05:32:19.913861 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:19.913885 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:19.914658 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0811 05:32:19.915654 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:19.920226 40506 executor.go:111] Tasks: 45 done / 105 total; 25 can run I0811 05:32:19.926219 40506 executor.go:111] Tasks: 70 done / 105 total; 31 can run I0811 05:32:19.926658 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:19.926823 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:19.927030 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:19.927259 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:19.928190 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:19.928970 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:19.929203 40506 executor.go:111] Tasks: 101 done / 105 total; 4 can run I0811 05:32:19.929916 40506 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0811 05:32:20.000709 40506 target.go:112] Terraform output is in /tmp/test2081428196/out --- PASS: TestUnmanaged (0.28s) === RUN TestPrivateSharedSubnet I0811 05:32:20.005103 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:20.005352 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:20.005414 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:20.005470 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:20.005526 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:20.005621 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:20.005651 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:20.005710 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:20.005746 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:20.034311 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:20.037209 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:20.037298 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:20.037356 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:20.074496 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:20.177413 40506 executor.go:111] Tasks: 0 done / 104 total; 45 can run I0811 05:32:20.177912 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:20.177950 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:20.178049 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0811 05:32:20.178166 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:20.179229 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:20.181031 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:20.181527 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:20.185276 40506 executor.go:111] Tasks: 45 done / 104 total; 24 can run I0811 05:32:20.185957 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0811 05:32:20.191054 40506 executor.go:111] Tasks: 69 done / 104 total; 31 can run I0811 05:32:20.191516 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:20.191794 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:20.192069 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:20.192376 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:20.192926 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:20.194421 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:20.194546 40506 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0811 05:32:20.194886 40506 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0811 05:32:20.264445 40506 target.go:112] Terraform output is in /tmp/test853440305/out --- PASS: TestPrivateSharedSubnet (0.26s) === RUN TestPrivateSharedIP I0811 05:32:20.268835 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:20.269098 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:20.269157 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:20.269209 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:20.269267 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:20.269364 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:20.269396 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:20.269434 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:20.269464 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:20.298089 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:20.304644 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:20.304743 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:20.304803 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:20.341968 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:20.445299 40506 executor.go:111] Tasks: 0 done / 113 total; 45 can run I0811 05:32:20.445782 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:20.446194 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:20.446379 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0811 05:32:20.446461 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:20.446420 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:20.446536 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0811 05:32:20.446547 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:20.448535 40506 executor.go:111] Tasks: 45 done / 113 total; 26 can run I0811 05:32:20.449073 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0811 05:32:20.454252 40506 executor.go:111] Tasks: 71 done / 113 total; 36 can run I0811 05:32:20.454562 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:20.454907 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:20.455161 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:20.455324 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:20.455852 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:20.457869 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:20.458380 40506 executor.go:111] Tasks: 107 done / 113 total; 5 can run I0811 05:32:20.459156 40506 executor.go:111] Tasks: 112 done / 113 total; 1 can run I0811 05:32:20.459388 40506 executor.go:111] Tasks: 113 done / 113 total; 0 can run I0811 05:32:20.534353 40506 target.go:112] Terraform output is in /tmp/test1641243514/out I0811 05:32:20.538530 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:20.538750 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:20.538807 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:20.538858 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:20.538913 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:20.538957 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:20.538987 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:20.539044 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:20.539078 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:20.567402 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:20.570458 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:20.570558 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:20.570625 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:20.608339 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0811 05:32:20.710411 40506 executor.go:111] Tasks: 0 done / 113 total; 45 can run I0811 05:32:20.710972 40506 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0811 05:32:20.711015 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:20.711171 40506 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0811 05:32:20.711319 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0811 05:32:20.711367 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:20.711380 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:20.711560 40506 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key W0811 05:32:20.711438 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:20.711585 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:20.711704 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0811 05:32:20.716925 40506 executor.go:111] Tasks: 45 done / 113 total; 26 can run I0811 05:32:20.717475 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0811 05:32:20.722403 40506 executor.go:111] Tasks: 71 done / 113 total; 36 can run I0811 05:32:20.723273 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:20.723499 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:20.723619 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:20.723867 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:20.724968 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:20.725908 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:20.726061 40506 executor.go:111] Tasks: 107 done / 113 total; 5 can run I0811 05:32:20.726418 40506 executor.go:111] Tasks: 112 done / 113 total; 1 can run I0811 05:32:20.726573 40506 executor.go:111] Tasks: 113 done / 113 total; 0 can run I0811 05:32:20.730650 40506 target.go:168] Cloudformation output is in /tmp/test1193093494/out --- PASS: TestPrivateSharedIP (0.48s) === RUN TestPrivateDns1 I0811 05:32:20.744945 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:20.745487 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:20.745613 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:20.745745 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:20.745861 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:20.746073 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:20.746167 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:20.746228 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:20.746321 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:20.779077 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:20.779178 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:20.779258 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:20.933052 40506 executor.go:111] Tasks: 0 done / 117 total; 46 can run W0811 05:32:20.933514 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:20.933742 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:20.933912 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns1.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:20.933978 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:20.941210 40506 executor.go:111] Tasks: 46 done / 117 total; 24 can run I0811 05:32:20.941695 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "internal.example.com" I0811 05:32:20.941739 40506 zones.go:40] GetHostedZone { Id: "/hostedzone/Z2AFAKE1ZON3NO" } I0811 05:32:20.941833 40506 dnszone.go:249] Existing zone "internal.example.com." found; will configure TF to reuse I0811 05:32:20.941854 40506 dnszone.go:272] No association between VPC "privatedns1.example.com" and zone "internal.example.com."; adding I0811 05:32:20.942061 40506 executor.go:111] Tasks: 70 done / 117 total; 35 can run I0811 05:32:20.943060 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:20.943274 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:20.945585 40506 executor.go:111] Tasks: 105 done / 117 total; 5 can run W0811 05:32:20.945796 40506 api.go:53] PageSize not implemented W0811 05:32:20.945927 40506 api.go:53] PageSize not implemented I0811 05:32:20.950952 40506 executor.go:111] Tasks: 110 done / 117 total; 5 can run I0811 05:32:20.951228 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:20.951526 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:20.952969 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:20.954062 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:20.954194 40506 executor.go:111] Tasks: 115 done / 117 total; 2 can run I0811 05:32:20.954481 40506 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0811 05:32:21.040382 40506 target.go:112] Terraform output is in /tmp/test4097210180/out --- PASS: TestPrivateDns1 (0.30s) === RUN TestPrivateDns2 I0811 05:32:21.045083 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:21.045334 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:21.045401 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:21.045456 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:21.045513 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:21.045611 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:21.045643 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:21.045700 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:21.045738 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:21.077147 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:21.077241 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:21.077307 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:21.209557 40506 executor.go:111] Tasks: 0 done / 113 total; 44 can run W0811 05:32:21.210204 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:21.210541 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:21.210554 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns2.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:21.210681 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:21.213352 40506 executor.go:111] Tasks: 44 done / 113 total; 22 can run I0811 05:32:21.213930 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0811 05:32:21.213989 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "private.example.com" I0811 05:32:21.214065 40506 zones.go:40] GetHostedZone { Id: "/hostedzone/Z3AFAKE1ZOMORE" } I0811 05:32:21.214136 40506 dnszone.go:249] Existing zone "private.example.com." found; will configure TF to reuse I0811 05:32:21.214158 40506 dnszone.go:263] VPC "vpc-12345678" already associated with zone "private.example.com." I0811 05:32:21.214231 40506 executor.go:111] Tasks: 66 done / 113 total; 35 can run I0811 05:32:21.214447 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:21.217532 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:21.223268 40506 executor.go:111] Tasks: 101 done / 113 total; 5 can run W0811 05:32:21.223354 40506 api.go:53] PageSize not implemented W0811 05:32:21.223424 40506 api.go:53] PageSize not implemented I0811 05:32:21.228498 40506 executor.go:111] Tasks: 106 done / 113 total; 5 can run I0811 05:32:21.228765 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:21.228973 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:21.229906 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:21.231231 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:21.231350 40506 executor.go:111] Tasks: 111 done / 113 total; 2 can run I0811 05:32:21.231686 40506 executor.go:111] Tasks: 113 done / 113 total; 0 can run I0811 05:32:21.303502 40506 target.go:112] Terraform output is in /tmp/test4069916257/out --- PASS: TestPrivateDns2 (0.26s) === RUN TestDiscoveryFeatureGate I0811 05:32:21.307922 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:21.308172 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:21.308232 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:21.308287 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:21.308343 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:21.308447 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:21.308480 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:21.308517 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:21.308549 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:21.339790 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.19.16 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.19.16 ********************************************************************************* I0811 05:32:21.343153 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:21.343278 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:21.380283 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:21.491051 40506 executor.go:111] Tasks: 0 done / 90 total; 46 can run W0811 05:32:21.492061 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:21.492087 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:21.492134 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:21.492172 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:21.492270 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:21.492303 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:21.494054 40506 executor.go:111] Tasks: 46 done / 90 total; 21 can run I0811 05:32:21.503421 40506 executor.go:111] Tasks: 67 done / 90 total; 21 can run I0811 05:32:21.503794 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:21.504030 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:21.505134 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:21.506176 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:21.506305 40506 executor.go:111] Tasks: 88 done / 90 total; 2 can run I0811 05:32:21.506593 40506 executor.go:111] Tasks: 90 done / 90 total; 0 can run I0811 05:32:21.563391 40506 target.go:112] Terraform output is in /tmp/test306871987/out --- PASS: TestDiscoveryFeatureGate (0.26s) === RUN TestVFSServiceAccountIssuerDiscovery I0811 05:32:21.567872 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:21.568117 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:21.568173 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:21.568225 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:21.568279 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:21.568376 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:21.568406 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:21.568444 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:21.568474 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:21.592725 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:21.595731 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:21.595828 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:21.632770 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:21.729804 40506 executor.go:111] Tasks: 0 done / 88 total; 45 can run W0811 05:32:21.730157 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:21.730723 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0811 05:32:21.730757 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:21.730806 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:21.730776 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:21.730941 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:21.733225 40506 executor.go:111] Tasks: 45 done / 88 total; 20 can run I0811 05:32:21.738446 40506 executor.go:111] Tasks: 65 done / 88 total; 21 can run I0811 05:32:21.738681 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:21.738848 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:21.741610 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:21.744207 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:21.744351 40506 executor.go:111] Tasks: 86 done / 88 total; 2 can run I0811 05:32:21.744666 40506 executor.go:111] Tasks: 88 done / 88 total; 0 can run I0811 05:32:21.793953 40506 target.go:112] Terraform output is in /tmp/test4233246804/out --- PASS: TestVFSServiceAccountIssuerDiscovery (0.23s) === RUN TestAWSLBController I0811 05:32:21.798677 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:21.798924 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:21.798983 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:21.799061 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:21.799116 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:21.799208 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:21.799239 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:21.799277 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:21.799310 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:21.830316 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:21.833252 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:21.833348 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:21.870063 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:22.487399 40506 executor.go:111] Tasks: 0 done / 94 total; 49 can run W0811 05:32:22.487828 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:22.487941 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:22.488432 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:22.488513 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:22.488552 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:22.488687 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:22.492703 40506 executor.go:111] Tasks: 49 done / 94 total; 22 can run I0811 05:32:22.501772 40506 executor.go:111] Tasks: 71 done / 94 total; 21 can run I0811 05:32:22.502307 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:22.502566 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:22.503591 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:22.504996 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:22.505132 40506 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0811 05:32:22.505470 40506 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0811 05:32:22.563729 40506 target.go:112] Terraform output is in /tmp/test2357738057/out --- PASS: TestAWSLBController (0.77s) === RUN TestManyAddons I0811 05:32:22.571599 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:22.571837 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:22.571895 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:22.571949 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:22.572002 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:22.572098 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:22.572130 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:22.572170 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:22.572199 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:22.598068 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:22.601131 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:22.601214 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:22.638193 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:23.437184 40506 executor.go:111] Tasks: 0 done / 93 total; 50 can run W0811 05:32:23.437489 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:23.437779 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:23.437817 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:23.437964 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:23.438040 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:23.438309 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:23.447359 40506 executor.go:111] Tasks: 50 done / 93 total; 19 can run I0811 05:32:23.452946 40506 executor.go:111] Tasks: 69 done / 93 total; 22 can run I0811 05:32:23.453212 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:23.453446 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:23.454511 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:23.456008 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:23.456133 40506 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0811 05:32:23.456457 40506 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0811 05:32:23.510511 40506 target.go:112] Terraform output is in /tmp/test3606557707/out --- PASS: TestManyAddons (0.95s) === RUN TestManyAddonsCCMIRSA I0811 05:32:23.522081 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:23.522502 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:23.522561 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:23.522611 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:23.522666 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:23.522752 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:23.522783 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:23.522821 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:23.522850 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:23.550049 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:23.553693 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:23.553788 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:23.590857 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:24.461004 40506 executor.go:111] Tasks: 0 done / 109 total; 59 can run W0811 05:32:24.461473 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:24.461699 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:24.461780 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:24.461956 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:24.461989 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:24.462056 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:24.469106 40506 executor.go:111] Tasks: 59 done / 109 total; 26 can run I0811 05:32:24.475646 40506 executor.go:111] Tasks: 85 done / 109 total; 22 can run I0811 05:32:24.475952 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:24.476331 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:24.477199 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:24.478869 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:24.479005 40506 executor.go:111] Tasks: 107 done / 109 total; 2 can run I0811 05:32:24.480573 40506 executor.go:111] Tasks: 109 done / 109 total; 0 can run I0811 05:32:24.549122 40506 target.go:112] Terraform output is in /tmp/test2822078585/out --- PASS: TestManyAddonsCCMIRSA (1.04s) === RUN TestManyAddonsCCMIRSA23 I0811 05:32:24.558843 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:24.559115 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:24.559171 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:24.559222 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:24.559274 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:24.559363 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:24.559394 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:24.559431 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:24.559461 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:24.586496 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* I0811 05:32:24.589756 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:24.589840 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:24.629520 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:25.500412 40506 executor.go:111] Tasks: 0 done / 110 total; 60 can run W0811 05:32:25.500879 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:25.501210 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:25.501511 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:25.501802 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:25.501844 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:25.501985 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:25.507846 40506 executor.go:111] Tasks: 60 done / 110 total; 26 can run I0811 05:32:25.513198 40506 executor.go:111] Tasks: 86 done / 110 total; 22 can run I0811 05:32:25.513536 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:25.513800 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:25.514770 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:25.516785 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:25.516915 40506 executor.go:111] Tasks: 108 done / 110 total; 2 can run I0811 05:32:25.517262 40506 executor.go:111] Tasks: 110 done / 110 total; 0 can run I0811 05:32:25.588004 40506 target.go:112] Terraform output is in /tmp/test236378924/out --- PASS: TestManyAddonsCCMIRSA23 (1.04s) === RUN TestManyAddonsCCMIRSA24 I0811 05:32:25.597325 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:25.597570 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:25.597630 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:25.597683 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:25.597736 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:25.597826 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:25.597857 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:25.597896 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:25.597925 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:25.628650 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0811 05:32:25.632720 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:25.632820 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:25.672048 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:26.544538 40506 executor.go:111] Tasks: 0 done / 110 total; 60 can run I0811 05:32:26.545273 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:26.545311 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:26.545396 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:26.545538 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:26.545607 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:26.545654 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:26.549593 40506 executor.go:111] Tasks: 60 done / 110 total; 26 can run I0811 05:32:26.561228 40506 executor.go:111] Tasks: 86 done / 110 total; 22 can run I0811 05:32:26.561876 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:26.562092 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:26.563086 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:26.564624 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:26.564763 40506 executor.go:111] Tasks: 108 done / 110 total; 2 can run I0811 05:32:26.565143 40506 executor.go:111] Tasks: 110 done / 110 total; 0 can run I0811 05:32:26.634127 40506 target.go:112] Terraform output is in /tmp/test288743186/out --- PASS: TestManyAddonsCCMIRSA24 (1.05s) === RUN TestCCM I0811 05:32:26.643350 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:26.643599 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:26.643653 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:26.643702 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:26.643756 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:26.643844 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:26.643875 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:26.643913 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:26.643941 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:26.670231 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:26.673855 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:26.673940 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:26.710659 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:27.512147 40506 executor.go:111] Tasks: 0 done / 94 total; 51 can run W0811 05:32:27.512562 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:27.513297 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:27.513287 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:27.513342 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:27.513449 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:27.513531 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:27.520481 40506 executor.go:111] Tasks: 51 done / 94 total; 19 can run I0811 05:32:27.526016 40506 executor.go:111] Tasks: 70 done / 94 total; 22 can run I0811 05:32:27.526418 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:27.526593 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:27.527694 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:27.529141 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:27.529266 40506 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0811 05:32:27.529579 40506 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0811 05:32:27.586059 40506 target.go:112] Terraform output is in /tmp/test4255089835/out --- PASS: TestCCM (0.95s) === RUN TestExternalDNS I0811 05:32:27.593662 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:27.593913 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:27.593970 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:27.594023 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:27.594075 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:27.594165 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:27.594195 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:27.594232 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:27.594265 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:27.618242 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:27.621078 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:27.621174 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:27.659848 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:27.763133 40506 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0811 05:32:27.763602 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:27.763649 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:27.763673 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:27.763757 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0811 05:32:27.763861 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:27.764083 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:27.766561 40506 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0811 05:32:27.774827 40506 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0811 05:32:27.775321 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:27.775556 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:27.776520 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:27.777880 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:27.778004 40506 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0811 05:32:27.778328 40506 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0811 05:32:27.833372 40506 target.go:112] Terraform output is in /tmp/test2906448953/out I0811 05:32:27.837314 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:27.837540 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:27.837603 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:27.837659 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:27.837718 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:27.837767 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:27.837797 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:27.837839 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:27.837870 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:27.862771 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:27.865614 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:27.865772 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:27.902562 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0811 05:32:28.003637 40506 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0811 05:32:28.003869 40506 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0811 05:32:28.003892 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:28.003981 40506 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0811 05:32:28.004236 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0811 05:32:28.004316 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:28.004392 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:28.004594 40506 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0811 05:32:28.004624 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:28.004820 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0811 05:32:28.006949 40506 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0811 05:32:28.016513 40506 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0811 05:32:28.017075 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.017332 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.018482 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.019359 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.019493 40506 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0811 05:32:28.019771 40506 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0811 05:32:28.023327 40506 target.go:168] Cloudformation output is in /tmp/test1675706053/out --- PASS: TestExternalDNS (0.44s) === RUN TestExternalDNSIRSA I0811 05:32:28.033219 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:28.033451 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:28.033505 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:28.033554 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:28.033606 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:28.033688 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:28.033718 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:28.033758 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:28.033789 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:28.062301 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:28.065214 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.065309 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.102236 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:28.218088 40506 executor.go:111] Tasks: 0 done / 90 total; 46 can run I0811 05:32:28.218340 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:28.218372 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:28.218465 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0811 05:32:28.218625 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:28.219132 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:28.219157 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:28.221650 40506 executor.go:111] Tasks: 46 done / 90 total; 21 can run I0811 05:32:28.226537 40506 executor.go:111] Tasks: 67 done / 90 total; 21 can run I0811 05:32:28.226998 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.227249 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.228296 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.229420 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.229538 40506 executor.go:111] Tasks: 88 done / 90 total; 2 can run I0811 05:32:28.229851 40506 executor.go:111] Tasks: 90 done / 90 total; 0 can run I0811 05:32:28.283704 40506 target.go:112] Terraform output is in /tmp/test2283817305/out --- PASS: TestExternalDNSIRSA (0.25s) === RUN TestKarpenter I0811 05:32:28.288298 40506 featureflag.go:162] FeatureFlag "Karpenter"=true I0811 05:32:28.288340 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:28.288573 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:28.288634 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:28.288694 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:28.288751 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:28.288849 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:28.288887 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:28.288930 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:28.288962 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:28.324595 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:28.327950 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.328047 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.328110 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.366304 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:28.459550 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.459715 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.459830 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.554234 40506 executor.go:111] Tasks: 0 done / 101 total; 50 can run W0811 05:32:28.554647 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:28.554781 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:28.554811 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:28.555070 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:28.555124 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:28.555271 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:28.555254 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:28.555317 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:28.557787 40506 executor.go:111] Tasks: 50 done / 101 total; 24 can run I0811 05:32:28.569352 40506 executor.go:111] Tasks: 74 done / 101 total; 25 can run I0811 05:32:28.569681 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.570023 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.570172 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.570374 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.570977 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.571247 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.571365 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.572762 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.572890 40506 executor.go:111] Tasks: 99 done / 101 total; 2 can run I0811 05:32:28.573346 40506 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0811 05:32:28.640893 40506 target.go:112] Terraform output is in /tmp/test3709407845/out I0811 05:32:28.646349 40506 featureflag.go:162] FeatureFlag "Karpenter"=false --- PASS: TestKarpenter (0.36s) === RUN TestSharedSubnet I0811 05:32:28.646477 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:28.646717 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:28.646773 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:28.646826 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:28.646879 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:28.646982 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:28.647017 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:28.647057 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:28.647089 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:28.671774 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:28.674675 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.674773 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.711310 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:28.808380 40506 executor.go:111] Tasks: 0 done / 78 total; 42 can run W0811 05:32:28.808853 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:28.808892 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:28.809056 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:28.809439 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:28.809486 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:28.809621 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:28.812024 40506 executor.go:111] Tasks: 42 done / 78 total; 16 can run I0811 05:32:28.812471 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0811 05:32:28.821811 40506 executor.go:111] Tasks: 58 done / 78 total; 18 can run I0811 05:32:28.822332 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.822628 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.823709 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.824659 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.824779 40506 executor.go:111] Tasks: 76 done / 78 total; 2 can run I0811 05:32:28.825121 40506 executor.go:111] Tasks: 78 done / 78 total; 0 can run I0811 05:32:28.867916 40506 target.go:112] Terraform output is in /tmp/test3227808014/out --- PASS: TestSharedSubnet (0.23s) === RUN TestSharedVPC I0811 05:32:28.872441 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:28.872689 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:28.872746 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:28.872799 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:28.872860 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:28.872974 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:28.873006 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:28.873045 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:28.873075 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:28.901065 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:28.904087 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.904191 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:28.941004 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:29.039490 40506 executor.go:111] Tasks: 0 done / 82 total; 42 can run W0811 05:32:29.039967 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:29.039984 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:29.040021 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:29.040111 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0811 05:32:29.040225 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:29.040421 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:29.042608 40506 executor.go:111] Tasks: 42 done / 82 total; 17 can run I0811 05:32:29.043123 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0811 05:32:29.048149 40506 executor.go:111] Tasks: 59 done / 82 total; 21 can run I0811 05:32:29.048522 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:29.048753 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:29.049869 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:29.050883 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:29.051015 40506 executor.go:111] Tasks: 80 done / 82 total; 2 can run I0811 05:32:29.051324 40506 executor.go:111] Tasks: 82 done / 82 total; 0 can run I0811 05:32:29.101458 40506 target.go:112] Terraform output is in /tmp/test1843508610/out --- PASS: TestSharedVPC (0.23s) === RUN TestExistingIAM I0811 05:32:29.105370 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:29.105608 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:29.105665 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:29.105738 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:29.105795 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:29.105884 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:29.105914 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:29.105951 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:29.105981 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:29.138189 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:29.141198 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:29.141291 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:29.141348 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:29.141406 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:29.178695 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:29.278843 40506 executor.go:111] Tasks: 0 done / 93 total; 47 can run I0811 05:32:29.279217 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0811 05:32:29.279244 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:29.279249 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:29.279333 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0811 05:32:29.279495 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:29.279620 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:29.280949 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existing-iam.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:29.280082 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:29.286679 40506 executor.go:111] Tasks: 47 done / 93 total; 15 can run I0811 05:32:29.291979 40506 executor.go:111] Tasks: 62 done / 93 total; 27 can run I0811 05:32:29.292282 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:29.292698 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:29.292842 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:29.293100 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:29.295709 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:29.297154 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:29.300249 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:29.300365 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:29.300550 40506 executor.go:111] Tasks: 89 done / 93 total; 4 can run I0811 05:32:29.301057 40506 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0811 05:32:29.370552 40506 target.go:112] Terraform output is in /tmp/test926148270/out --- PASS: TestExistingIAM (0.27s) === RUN TestPhaseNetwork I0811 05:32:29.374867 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:29.375112 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:29.375170 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:29.375222 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:29.375276 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:29.375322 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:29.375351 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:29.375390 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:29.375421 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:29.403133 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:29.406175 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:29.406262 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:29.406321 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:29.443154 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:29.549055 40506 executor.go:111] Tasks: 0 done / 116 total; 46 can run I0811 05:32:29.549365 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.lifecyclephases.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:29.549518 40506 executor.go:111] Tasks: 46 done / 116 total; 28 can run I0811 05:32:29.550042 40506 executor.go:111] Tasks: 74 done / 116 total; 36 can run I0811 05:32:29.550474 40506 executor.go:111] Tasks: 110 done / 116 total; 5 can run I0811 05:32:29.550675 40506 executor.go:111] Tasks: 115 done / 116 total; 1 can run I0811 05:32:29.550813 40506 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0811 05:32:29.559221 40506 target.go:112] Terraform output is in /tmp/test3137186159/out --- PASS: TestPhaseNetwork (0.19s) === RUN TestExternalLoadBalancer I0811 05:32:29.559965 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:29.560189 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:29.560239 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:29.560285 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:29.560336 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:29.560424 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:29.560454 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:29.560493 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:29.560522 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:29.585027 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:29.587881 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:29.587987 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:29.625226 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:29.727752 40506 executor.go:111] Tasks: 0 done / 92 total; 50 can run I0811 05:32:29.728151 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:29.728178 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:29.728309 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0811 05:32:29.728577 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:29.728638 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:29.730586 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:29.735818 40506 executor.go:111] Tasks: 50 done / 92 total; 19 can run I0811 05:32:29.741276 40506 executor.go:111] Tasks: 69 done / 92 total; 21 can run I0811 05:32:29.741473 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:29.741721 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:29.742554 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:29.744412 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:29.744532 40506 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0811 05:32:29.744926 40506 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0811 05:32:29.798906 40506 target.go:112] Terraform output is in /tmp/test1308442688/out I0811 05:32:29.803462 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:29.803728 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:29.803785 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:29.803833 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:29.803901 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:29.803952 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:29.803985 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:29.804026 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:29.804056 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:29.828971 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:29.831868 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:29.831973 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:29.869346 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0811 05:32:29.970326 40506 executor.go:111] Tasks: 0 done / 92 total; 50 can run I0811 05:32:29.970799 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:29.970943 40506 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0811 05:32:29.970947 40506 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0811 05:32:29.970962 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:29.970985 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:29.971113 40506 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0811 05:32:29.971067 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } W0811 05:32:29.971252 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0811 05:32:29.971262 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:29.973194 40506 executor.go:111] Tasks: 50 done / 92 total; 19 can run I0811 05:32:29.982267 40506 executor.go:111] Tasks: 69 done / 92 total; 21 can run I0811 05:32:29.982716 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:29.982888 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:29.983891 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:29.985113 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:29.985252 40506 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0811 05:32:29.985522 40506 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0811 05:32:29.988945 40506 target.go:168] Cloudformation output is in /tmp/test4080199158/out --- PASS: TestExternalLoadBalancer (0.44s) === RUN TestPhaseIAM integration_test.go:750: unable to test w/o allowing failed validation --- SKIP: TestPhaseIAM (0.00s) === RUN TestPhaseCluster integration_test.go:758: unable to test w/o allowing failed validation --- SKIP: TestPhaseCluster (0.00s) === RUN TestMixedInstancesASG I0811 05:32:29.999329 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:29.999557 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:29.999608 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:29.999652 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:29.999702 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:29.999779 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:29.999810 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:29.999850 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:29.999879 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:30.036396 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:30.040157 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.040263 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.040372 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.040487 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.040599 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.040693 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.040755 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.076810 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:30.182132 40506 executor.go:111] Tasks: 0 done / 103 total; 49 can run W0811 05:32:30.182660 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:30.182991 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:30.183126 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:30.183174 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0811 05:32:30.183295 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:30.183137 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:30.183311 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:30.183387 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:30.185832 40506 executor.go:111] Tasks: 49 done / 103 total; 23 can run I0811 05:32:30.191333 40506 executor.go:111] Tasks: 72 done / 103 total; 27 can run I0811 05:32:30.191540 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.191686 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.191914 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.192089 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.196330 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.198541 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.199126 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.199336 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.199474 40506 executor.go:111] Tasks: 99 done / 103 total; 4 can run I0811 05:32:30.199900 40506 executor.go:111] Tasks: 103 done / 103 total; 0 can run I0811 05:32:30.272929 40506 target.go:112] Terraform output is in /tmp/test2488817726/out I0811 05:32:30.277480 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:30.277696 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:30.277755 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:30.277809 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:30.277862 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:30.277906 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:30.277935 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:30.277972 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:30.278000 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:30.316643 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:30.320061 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.320176 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.320264 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.320326 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.320383 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.320436 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.320484 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.358015 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0811 05:32:30.462255 40506 executor.go:111] Tasks: 0 done / 103 total; 49 can run W0811 05:32:30.462676 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0811 05:32:30.463131 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0811 05:32:30.463208 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:30.463291 40506 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0811 05:32:30.463326 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0811 05:32:30.463411 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:30.463421 40506 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0811 05:32:30.463584 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:30.463718 40506 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0811 05:32:30.463735 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:30.463822 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "mixedinstances.example.com" },{ Key: "KubernetesCluster", Value: "mixedinstances.example.com" },{ Key: "kubernetes.io/cluster/mixedinstances.example.com", Value: "owned" }] }] } I0811 05:32:30.465270 40506 executor.go:111] Tasks: 49 done / 103 total; 23 can run I0811 05:32:30.475488 40506 executor.go:111] Tasks: 72 done / 103 total; 27 can run I0811 05:32:30.475903 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.476210 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.476497 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.476804 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.477938 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.478671 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.479763 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.480135 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.480276 40506 executor.go:111] Tasks: 99 done / 103 total; 4 can run I0811 05:32:30.480743 40506 executor.go:111] Tasks: 103 done / 103 total; 0 can run I0811 05:32:30.486009 40506 target.go:168] Cloudformation output is in /tmp/test316409551/out --- PASS: TestMixedInstancesASG (0.51s) === RUN TestMixedInstancesSpotASG I0811 05:32:30.505905 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:30.506149 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:30.506201 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:30.506249 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:30.506301 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:30.506389 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:30.506419 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:30.506456 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:30.506485 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:30.538763 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:30.541915 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.542008 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.542072 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.542133 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.542188 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.542242 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.542293 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.578125 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:30.679884 40506 executor.go:111] Tasks: 0 done / 103 total; 49 can run I0811 05:32:30.680493 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:30.680548 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0811 05:32:30.680688 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:30.680693 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0811 05:32:30.680858 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:30.680863 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:30.680954 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:30.681033 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:30.683302 40506 executor.go:111] Tasks: 49 done / 103 total; 23 can run I0811 05:32:30.694055 40506 executor.go:111] Tasks: 72 done / 103 total; 27 can run I0811 05:32:30.694499 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.694707 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.694993 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.695285 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.696527 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.697002 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.697743 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.697938 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.698063 40506 executor.go:111] Tasks: 99 done / 103 total; 4 can run I0811 05:32:30.698479 40506 executor.go:111] Tasks: 103 done / 103 total; 0 can run I0811 05:32:30.776512 40506 target.go:112] Terraform output is in /tmp/test335712381/out I0811 05:32:30.781167 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:30.781410 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:30.781473 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:30.781531 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:30.781586 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:30.781633 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:30.781666 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:30.781710 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:30.781742 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:30.814704 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:30.817672 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.817759 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.817812 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.817870 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.817920 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.817966 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.818010 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.857172 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0811 05:32:30.957775 40506 executor.go:111] Tasks: 0 done / 103 total; 49 can run W0811 05:32:30.959543 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0811 05:32:30.960315 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:30.960482 40506 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0811 05:32:30.960581 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:30.960528 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:30.960662 40506 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0811 05:32:30.960747 40506 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0811 05:32:30.960773 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:30.960749 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0811 05:32:30.960888 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:30.960938 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "mixedinstances.example.com" },{ Key: "KubernetesCluster", Value: "mixedinstances.example.com" },{ Key: "kubernetes.io/cluster/mixedinstances.example.com", Value: "owned" }] }] } I0811 05:32:30.964936 40506 executor.go:111] Tasks: 49 done / 103 total; 23 can run I0811 05:32:30.970618 40506 executor.go:111] Tasks: 72 done / 103 total; 27 can run I0811 05:32:30.970969 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.971178 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.971403 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.971678 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.974205 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.976320 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.976755 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.977965 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:30.978117 40506 executor.go:111] Tasks: 99 done / 103 total; 4 can run I0811 05:32:30.978531 40506 executor.go:111] Tasks: 103 done / 103 total; 0 can run I0811 05:32:30.983194 40506 target.go:168] Cloudformation output is in /tmp/test1720678218/out --- PASS: TestMixedInstancesSpotASG (0.49s) === RUN TestContainerd I0811 05:32:30.999508 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:30.999746 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:30.999800 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:30.999893 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:30.999946 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:30.999990 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:31.000020 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:31.000063 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:31.000091 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:31.023858 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:31.026710 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:31.026795 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:31.063902 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0811 05:32:31.165611 40506 executor.go:111] Tasks: 0 done / 85 total; 43 can run W0811 05:32:31.165873 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:31.166110 40506 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0811 05:32:31.166146 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:31.166239 40506 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0811 05:32:31.166306 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:31.166448 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0811 05:32:31.166500 40506 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0811 05:32:31.166529 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:31.166717 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/containerd.example.com", Value: "owned" },{ Key: "Name", Value: "containerd.example.com" },{ Key: "KubernetesCluster", Value: "containerd.example.com" }] }] } I0811 05:32:31.168928 40506 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0811 05:32:31.177694 40506 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0811 05:32:31.178066 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:31.178359 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:31.179240 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:31.180599 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:31.180737 40506 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0811 05:32:31.180993 40506 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0811 05:32:31.183894 40506 target.go:168] Cloudformation output is in /tmp/test1350131101/out --- PASS: TestContainerd (0.19s) === RUN TestContainerdCustom I0811 05:32:31.194336 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:31.194586 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:31.194638 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:31.194748 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:31.194821 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:31.194873 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:31.194922 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:31.194976 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:31.195014 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:31.220018 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:31.223941 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:31.224141 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:31.261234 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0811 05:32:31.361718 40506 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0811 05:32:31.362161 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:31.362380 40506 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0811 05:32:31.362437 40506 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0811 05:32:31.362479 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0811 05:32:31.362502 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:31.362410 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:31.362573 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "containerd.example.com" },{ Key: "KubernetesCluster", Value: "containerd.example.com" },{ Key: "kubernetes.io/cluster/containerd.example.com", Value: "owned" }] }] } I0811 05:32:31.362463 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:31.362756 40506 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0811 05:32:31.365056 40506 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0811 05:32:31.370317 40506 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0811 05:32:31.370563 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:31.370771 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:31.372022 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:31.373138 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:31.373281 40506 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0811 05:32:31.373546 40506 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0811 05:32:31.376747 40506 target.go:168] Cloudformation output is in /tmp/test1718208632/out --- PASS: TestContainerdCustom (0.20s) === RUN TestDockerCustom I0811 05:32:31.390836 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:31.391075 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:31.391129 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:31.391220 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:31.391272 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:31.391314 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:31.391343 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:31.391383 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:31.391410 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:31.417108 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:31.420222 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:31.420318 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:31.458478 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0811 05:32:31.560484 40506 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0811 05:32:31.560911 40506 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0811 05:32:31.560956 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0811 05:32:31.561044 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:31.561060 40506 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0811 05:32:31.561253 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:31.561317 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:31.561440 40506 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0811 05:32:31.561459 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:31.561545 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "docker.example.com" },{ Key: "KubernetesCluster", Value: "docker.example.com" },{ Key: "kubernetes.io/cluster/docker.example.com", Value: "owned" }] }] } I0811 05:32:31.563810 40506 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0811 05:32:31.569102 40506 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0811 05:32:31.569476 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:31.569700 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:31.575455 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:31.576358 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:31.576519 40506 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0811 05:32:31.576797 40506 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0811 05:32:31.579848 40506 target.go:168] Cloudformation output is in /tmp/test1640932849/out --- PASS: TestDockerCustom (0.20s) === RUN TestAPIServerNodes I0811 05:32:31.590505 40506 featureflag.go:162] FeatureFlag "APIServerNodes"=true I0811 05:32:31.590547 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:31.590772 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:31.590826 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:31.590912 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:31.590964 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:31.591007 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:31.591038 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:31.591077 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:31.591107 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:31.617119 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.10 ********************************************************************************* I0811 05:32:31.620101 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:31.620189 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:31.620252 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:31.658949 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0811 05:32:31.816839 40506 executor.go:111] Tasks: 0 done / 97 total; 47 can run W0811 05:32:31.817653 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0811 05:32:31.817707 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:31.817774 40506 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0811 05:32:31.817879 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:31.817809 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:31.817959 40506 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0811 05:32:31.817946 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:31.818089 40506 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0811 05:32:31.818111 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:31.818203 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0811 05:32:31.819528 40506 executor.go:111] Tasks: 47 done / 97 total; 24 can run I0811 05:32:31.829110 40506 executor.go:111] Tasks: 71 done / 97 total; 23 can run I0811 05:32:31.829645 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:31.829860 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:31.830092 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:31.831626 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:31.831750 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:31.832163 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:31.832301 40506 executor.go:111] Tasks: 94 done / 97 total; 3 can run I0811 05:32:31.832645 40506 executor.go:111] Tasks: 97 done / 97 total; 0 can run I0811 05:32:31.836779 40506 target.go:168] Cloudformation output is in /tmp/test1276220399/out I0811 05:32:31.849526 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:31.849748 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:31.849800 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:31.849844 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:31.849897 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:31.849941 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:31.849972 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:31.850009 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:31.850038 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:31.873846 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.10 ********************************************************************************* I0811 05:32:31.877616 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:31.878005 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:31.878256 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:31.914776 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:32.071409 40506 executor.go:111] Tasks: 0 done / 97 total; 47 can run W0811 05:32:32.072182 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:32.073215 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:32.073875 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:32.074335 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:32.074377 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:32.074488 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:32.074671 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:32.080036 40506 executor.go:111] Tasks: 47 done / 97 total; 24 can run I0811 05:32:32.086213 40506 executor.go:111] Tasks: 71 done / 97 total; 23 can run I0811 05:32:32.086439 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:32.086808 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:32.087032 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:32.088553 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:32.088657 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:32.089510 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:32.089628 40506 executor.go:111] Tasks: 94 done / 97 total; 3 can run I0811 05:32:32.089963 40506 executor.go:111] Tasks: 97 done / 97 total; 0 can run I0811 05:32:32.154722 40506 target.go:112] Terraform output is in /tmp/test1558825255/out I0811 05:32:32.159326 40506 featureflag.go:162] FeatureFlag "APIServerNodes"=false --- PASS: TestAPIServerNodes (0.57s) === RUN TestNTHQueueProcessor I0811 05:32:32.159472 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:32.159736 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:32.159796 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:32.159852 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:32.159910 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:32.160015 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:32.160046 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:32.160090 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:32.160122 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:32.180852 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0811 05:32:32.183072 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:32.183151 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:32.221036 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:32.348085 40506 executor.go:111] Tasks: 0 done / 99 total; 45 can run I0811 05:32:32.348418 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:32.348531 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:32.348622 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:32.348661 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:32.348756 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0811 05:32:32.349210 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:32.350982 40506 executor.go:111] Tasks: 45 done / 99 total; 24 can run I0811 05:32:32.359700 40506 executor.go:111] Tasks: 69 done / 99 total; 26 can run I0811 05:32:32.359942 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:32.360283 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:32.360992 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:32.362576 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:32.362701 40506 executor.go:111] Tasks: 95 done / 99 total; 2 can run I0811 05:32:32.363060 40506 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0811 05:32:32.363364 40506 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0811 05:32:32.426759 40506 target.go:112] Terraform output is in /tmp/test3139470361/out I0811 05:32:32.431613 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:32.431833 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:32.431890 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:32.431942 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:32.431997 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:32.432043 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:32.432073 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:32.432112 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:32.432142 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:32.456648 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0811 05:32:32.459673 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:32.459770 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:32.496824 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0811 05:32:32.617791 40506 executor.go:111] Tasks: 0 done / 99 total; 45 can run I0811 05:32:32.618052 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:32.618169 40506 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0811 05:32:32.618200 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:32.618339 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } W0811 05:32:32.618576 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:32.620056 40506 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0811 05:32:32.620116 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:32.620220 40506 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0811 05:32:32.621768 40506 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0811 05:32:32.626378 40506 executor.go:111] Tasks: 45 done / 99 total; 24 can run I0811 05:32:32.631875 40506 executor.go:111] Tasks: 69 done / 99 total; 26 can run I0811 05:32:32.632166 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:32.632292 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:32.633347 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:32.634374 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:32.634516 40506 executor.go:111] Tasks: 95 done / 99 total; 2 can run I0811 05:32:32.634823 40506 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0811 05:32:32.635054 40506 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0811 05:32:32.639236 40506 target.go:168] Cloudformation output is in /tmp/test826565317/out --- PASS: TestNTHQueueProcessor (0.49s) === RUN TestCustomIRSA I0811 05:32:32.653405 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:32.653685 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:32.653736 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:32.653786 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:32.653836 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:32.653920 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:32.653955 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:32.653993 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:32.654023 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:32.683697 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0811 05:32:32.686998 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:32.687092 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:32.723960 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:33.277997 40506 executor.go:111] Tasks: 0 done / 98 total; 50 can run W0811 05:32:33.278887 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:33.278839 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:33.279082 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:33.279116 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:33.279154 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:33.282114 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:33.287396 40506 executor.go:111] Tasks: 50 done / 98 total; 25 can run I0811 05:32:33.294042 40506 executor.go:111] Tasks: 75 done / 98 total; 21 can run I0811 05:32:33.294321 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:33.294658 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:33.295445 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:33.297417 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:33.297547 40506 executor.go:111] Tasks: 96 done / 98 total; 2 can run I0811 05:32:33.297867 40506 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0811 05:32:33.354419 40506 target.go:112] Terraform output is in /tmp/test3956695666/out --- PASS: TestCustomIRSA (0.71s) === RUN TestCustomIRSA119 I0811 05:32:33.365571 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:33.365928 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:33.366023 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:33.366109 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:33.366202 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:33.366341 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:33.366401 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:33.366474 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:33.366525 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:33.394191 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.19.16 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.19.16 ********************************************************************************* I0811 05:32:33.397628 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:33.397722 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:33.435424 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:33.536576 40506 executor.go:111] Tasks: 0 done / 96 total; 48 can run W0811 05:32:33.537013 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target W0811 05:32:33.537085 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:33.537083 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:33.537779 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:33.537814 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:33.537910 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0811 05:32:33.539420 40506 executor.go:111] Tasks: 48 done / 96 total; 25 can run I0811 05:32:33.547953 40506 executor.go:111] Tasks: 73 done / 96 total; 21 can run I0811 05:32:33.548495 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:33.548706 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:33.549683 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:33.551032 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:33.551157 40506 executor.go:111] Tasks: 94 done / 96 total; 2 can run I0811 05:32:33.551447 40506 executor.go:111] Tasks: 96 done / 96 total; 0 can run I0811 05:32:33.610611 40506 target.go:112] Terraform output is in /tmp/test1743849027/out --- PASS: TestCustomIRSA119 (0.25s) === RUN TestClusterNameDigit I0811 05:32:33.615255 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:33.615495 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:33.615551 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:33.615601 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:33.615654 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:33.615766 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:33.615798 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:33.615836 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:33.615866 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:33.642777 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0811 05:32:33.645880 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:33.645979 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:33.684279 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:33.786645 40506 executor.go:111] Tasks: 0 done / 93 total; 47 can run I0811 05:32:33.787529 40506 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0811 05:32:33.787562 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:33.787653 40506 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0811 05:32:33.787530 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:33.787716 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.123.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0811 05:32:33.787788 40506 warmpool.go:124] ASG warm pool is not supported by the terraform target I0811 05:32:33.789557 40506 executor.go:111] Tasks: 47 done / 93 total; 23 can run I0811 05:32:33.799809 40506 executor.go:111] Tasks: 70 done / 93 total; 21 can run I0811 05:32:33.800179 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:33.800562 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:33.801853 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:33.802752 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:33.802883 40506 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0811 05:32:33.803197 40506 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0811 05:32:33.859378 40506 target.go:112] Terraform output is in /tmp/test762741088/out --- PASS: TestClusterNameDigit (0.25s) === RUN TestLifecycleMinimalAWS I0811 05:32:33.863981 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:33.864231 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:33.864294 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:33.864348 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:33.864411 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:33.864462 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:33.864492 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:33.864532 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:33.864563 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:33.864625 40506 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal.example.com I0811 05:32:33.888213 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:33.891080 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:33.891180 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0811 05:32:33.928121 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:34.024159 40506 iamrole.go:89] ListRoles: { } I0811 05:32:34.025781 40506 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0811 05:32:34.026034 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0811 05:32:34.026179 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:34.026281 40506 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0811 05:32:34.026505 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0811 05:32:34.026607 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] }] } I0811 05:32:34.026772 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:34.026897 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:34.026899 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I0811 05:32:34.027088 40506 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0811 05:32:34.027206 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } I0811 05:32:34.027278 40506 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0811 05:32:34.027146 40506 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0811 05:32:34.027465 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0811 05:32:34.027568 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0811 05:32:34.027876 40506 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } I0811 05:32:34.027882 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0811 05:32:34.028163 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0811 05:32:34.028382 40506 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0811 05:32:34.028432 40506 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0811 05:32:34.028468 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0811 05:32:34.028526 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:34.028568 40506 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0811 05:32:34.028699 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0811 05:32:34.028898 40506 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0811 05:32:34.029080 40506 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0811 05:32:34.029122 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0811 05:32:34.029185 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:34.029295 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0811 05:32:34.029339 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0811 05:32:34.029527 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0811 05:32:34.029536 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0811 05:32:34.029748 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:34.029780 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0811 05:32:34.029911 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:32:34.029968 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:32:34.030010 40506 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0811 05:32:34.030073 40506 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0811 05:32:34.030059 40506 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0811 05:32:34.030125 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0811 05:32:34.030234 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0811 05:32:34.030370 40506 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0811 05:32:34.030342 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0811 05:32:34.030565 40506 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0811 05:32:34.030658 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0811 05:32:34.030712 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0811 05:32:34.030871 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0811 05:32:34.031617 40506 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal.example.com" } ] } I0811 05:32:34.031892 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }], VpcId: "vpc-1" } I0811 05:32:34.032050 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0811 05:32:34.032165 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0811 05:32:34.032251 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:34.032307 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0811 05:32:34.032424 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:34.032871 40506 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0811 05:32:34.037185 40506 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0811 05:32:34.037473 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0811 05:32:34.037679 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0811 05:32:34.037867 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0811 05:32:34.038133 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0811 05:32:34.038297 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0811 05:32:34.038542 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-1to2379-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0811 05:32:34.038705 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0811 05:32:34.038882 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:34.038991 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:34.039116 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:34.039217 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:34.039330 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0811 05:32:34.039430 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0811 05:32:34.039497 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-2382to4000-masters.minimal.example.com" }] }] } I0811 05:32:34.039692 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:34.039830 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:34.039972 40506 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0811 05:32:34.040102 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0811 05:32:34.040361 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-4003to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0811 05:32:34.040539 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0811 05:32:34.040714 40506 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0811 05:32:34.040777 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-udp-1to65535-masters.minimal.example.com" }] }] } I0811 05:32:34.040973 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0811 05:32:34.041137 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:34.041285 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0811 05:32:34.043612 40506 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0811 05:32:34.044151 40506 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0811 05:32:34.044232 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:34.044272 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:34.044338 40506 dns.go:238] Pre-creating DNS records I0811 05:32:34.044367 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:34.050698 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:34.053522 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:34.053624 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0811 05:32:34.091322 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:34.189990 40506 iamrole.go:89] ListRoles: { } I0811 05:32:34.191638 40506 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0811 05:32:34.192149 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0811 05:32:34.192301 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:34.192314 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0811 05:32:34.192639 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0811 05:32:34.192873 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0811 05:32:34.193072 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0811 05:32:34.193148 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0811 05:32:34.193265 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:34.193340 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:34.193435 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:34.194605 40506 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0811 05:32:34.194730 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["minimal.example.com"] }] } I0811 05:32:34.194803 40506 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0811 05:32:34.194866 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:34.195007 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0811 05:32:34.194992 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0811 05:32:34.195185 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:34.195261 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0811 05:32:34.195420 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:34.195478 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0811 05:32:34.195661 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:32:34.195739 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["minimal.example.com"] }] } I0811 05:32:34.195876 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:34.199878 40506 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0811 05:32:34.200113 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:34.200236 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:34.200397 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:34.200516 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:34.200631 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:34.200815 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:34.201139 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:34.203249 40506 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0811 05:32:34.203460 40506 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0811 05:32:34.206277 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0811 05:32:34.206346 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0811 05:32:34.206388 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0811 05:32:34.206426 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0811 05:32:34.206467 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0811 05:32:34.206502 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0811 05:32:34.206535 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0811 05:32:34.206569 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0811 05:32:34.206606 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0811 05:32:34.206643 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0811 05:32:34.206680 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0811 05:32:34.206723 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0811 05:32:34.206758 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0811 05:32:34.206793 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0811 05:32:34.206830 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0811 05:32:34.206864 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0811 05:32:34.206902 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0811 05:32:34.206940 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:32:34.206979 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0811 05:32:34.207015 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0811 05:32:34.207062 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0811 05:32:34.207097 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0811 05:32:34.207134 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0811 05:32:34.207169 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0811 05:32:34.207205 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0811 05:32:34.208476 40506 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0811 05:32:34.208509 40506 keypairs.go:95] DescribeKeyPairs: { } I0811 05:32:34.208551 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0811 05:32:34.208606 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0811 05:32:34.208665 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0811 05:32:34.208727 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0811 05:32:34.208775 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0811 05:32:34.208831 40506 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0811 05:32:34.208881 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0811 05:32:34.208922 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0811 05:32:34.208971 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0811 05:32:34.209042 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0811 05:32:34.209086 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } W0811 05:32:34.209139 40506 tags.go:48] unknown cluster tag on vpc:vpc-12345678: "kubernetes.io/cluster/minimal.example.com"="" W0811 05:32:34.209173 40506 api.go:53] PageSize not implemented I0811 05:32:34.209195 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0811 05:32:34.209228 40506 loadbalancers.go:34] PageSize not implemented I0811 05:32:34.209249 40506 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0811 05:32:34.209277 40506 targetgroups.go:35] PageSize not implemented I0811 05:32:34.209295 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0811 05:32:34.209356 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:34.209387 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:34.209437 40506 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0811 05:32:34.209478 40506 iamrole.go:89] ListRoles: { } I0811 05:32:34.209549 40506 internetgateways.go:101] DescribeInternetGateways: { } I0811 05:32:34.209597 40506 routetable.go:53] DescribeRouteTables: { } I0811 05:32:34.209634 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:nodes.minimal.example.com ok I0811 05:32:34.210256 40506 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } autoscaling-config:lt-2 ok autoscaling-config:lt-1 ok volume:vol-1 ok I0811 05:32:34.210400 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:34.210362 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.minimal.example.com. ok I0811 05:32:34.210515 40506 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0811 05:32:34.210618 40506 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } iam-instance-profile:masters.minimal.example.com ok I0811 05:32:34.210661 40506 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } autoscaling-group:master-us-test-1a.masters.minimal.example.com ok I0811 05:32:34.210754 40506 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } iam-instance-profile:nodes.minimal.example.com ok I0811 05:32:34.210481 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0811 05:32:34.210808 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:34.210835 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0811 05:32:34.210878 40506 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0811 05:32:34.210948 40506 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0811 05:32:34.210989 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:34.211047 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0811 05:32:34.211177 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:34.211192 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0811 05:32:34.211235 40506 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0811 05:32:34.211299 40506 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0811 05:32:34.211403 40506 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal.example.com" } I0811 05:32:34.211464 40506 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal.example.com I0811 05:32:34.211429 40506 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } I0811 05:32:34.211482 40506 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } subnet:subnet-1 ok I0811 05:32:34.211518 40506 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal.example.com" } iam-role:nodes.minimal.example.com ok I0811 05:32:34.211557 40506 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal.example.com" } I0811 05:32:34.211600 40506 iamrole.go:180] ListAttachedRolePolicies: masters.minimal.example.com I0811 05:32:34.211613 40506 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0811 05:32:34.211641 40506 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal.example.com" } iam-role:masters.minimal.example.com ok I0811 05:32:34.211705 40506 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0811 05:32:34.211791 40506 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0811 05:32:34.211873 40506 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0811 05:32:34.212189 40506 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleMinimalAWS (0.35s) === RUN TestLifecycleMinimalOpenstack I0811 05:32:34.212468 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster minimal-openstack.k8s.local I0811 05:32:34.221843 40506 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:34.289682 40506 apply_cluster.go:467] Gossip DNS: skipping DNS validation W0811 05:32:34.443171 40506 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0811 05:32:34.443225 40506 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0811 05:32:34.462719 40506 executor.go:111] Tasks: 0 done / 96 total; 42 can run I0811 05:32:34.467596 40506 executor.go:111] Tasks: 42 done / 96 total; 45 can run I0811 05:32:34.500252 40506 executor.go:111] Tasks: 87 done / 96 total; 5 can run I0811 05:32:34.504115 40506 executor.go:111] Tasks: 92 done / 96 total; 4 can run W0811 05:32:34.507595 40506 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0811 05:32:34.507796 40506 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0811 05:32:34.507804 40506 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I0811 05:32:34.517650 40506 executor.go:111] Tasks: 96 done / 96 total; 0 can run ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:34.562073 40506 apply_cluster.go:467] Gossip DNS: skipping DNS validation W0811 05:32:34.706998 40506 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0811 05:32:34.707045 40506 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0811 05:32:34.726124 40506 executor.go:111] Tasks: 0 done / 96 total; 42 can run I0811 05:32:34.735126 40506 executor.go:111] Tasks: 42 done / 96 total; 45 can run I0811 05:32:34.746787 40506 executor.go:111] Tasks: 87 done / 96 total; 5 can run I0811 05:32:34.749360 40506 executor.go:111] Tasks: 92 done / 96 total; 4 can run W0811 05:32:34.752993 40506 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0811 05:32:34.753168 40506 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0811 05:32:34.753651 40506 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I0811 05:32:34.755153 40506 executor.go:111] Tasks: 96 done / 96 total; 0 can run SecurityGroup:9f53fa86-c848-49d2-bfc5-86edb150fa09 ok Instance:92970a17-e999-4c34-bef7-ea18741876d2 ok ServerGroup:3a8ec537-3707-43ac-946b-35eb56b5b74e ok Network:8901e969-ffa8-4666-ba94-1de128a83a40 ok SSHKey:kubernetes-minimal-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok Subnet:f443f485-f5a3-44c2-ae0f-fd18096ea58f ok Instance:7ea2d15d-2714-4805-9bc8-95fd870357aa ok SecurityGroup:eaee0a11-ad80-47a2-b35f-e6348eea167b ok Port:185758f8-7579-49cf-9d72-fd2fe41cf0e1 ok Port:b8f751af-0c06-4297-8b0c-d0c9a259b7e3 ok Port:3015a546-1d90-4eaa-b5f6-8049afba493b ok SecurityGroup:a8e93811-5738-47fc-b480-f14b55d79702 ok Instance:9098adf5-3ecc-44f2-93ab-8ff1aac116f0 ok ServerGroup:8b0c9f49-f5d5-40d1-8c3d-2cb321745901 ok --- PASS: TestLifecycleMinimalOpenstack (1.66s) === RUN TestLifecycleMinimalGCE I0811 05:32:35.871783 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:486: running lifecycle test for cluster minimal-gce.example.com I0811 05:32:35.873770 40506 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* W0811 05:32:36.112933 40506 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0811 05:32:36.113034 40506 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0811 05:32:36.113161 40506 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0811 05:32:36.113200 40506 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0811 05:32:36.113226 40506 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0811 05:32:36.113267 40506 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0811 05:32:36.113361 40506 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0811 05:32:36.115202 40506 executor.go:111] Tasks: 0 done / 64 total; 41 can run W0811 05:32:36.118138 40506 executor.go:139] error running task "ProjectIAMBinding/serviceaccount-control-plane" (9s remaining to succeed): error updating IAM for project testproject: googleapi: Error 404: etag I0811 05:32:36.118214 40506 executor.go:111] Tasks: 40 done / 64 total; 18 can run I0811 05:32:36.124145 40506 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0811 05:32:36.126800 40506 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0811 05:32:36.127037 40506 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0811 05:32:36.127133 40506 dns.go:238] Pre-creating DNS records ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* W0811 05:32:36.327470 40506 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0811 05:32:36.327563 40506 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0811 05:32:36.327667 40506 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0811 05:32:36.327704 40506 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0811 05:32:36.327730 40506 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0811 05:32:36.327763 40506 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0811 05:32:36.327834 40506 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0811 05:32:36.329532 40506 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0811 05:32:36.332765 40506 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0811 05:32:36.338168 40506 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0811 05:32:36.340754 40506 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0811 05:32:36.340926 40506 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0811 05:32:36.342684 40506 gce.go:98] Scanning zones: [us-test1-a] I0811 05:32:36.343051 40506 gce.go:1070] backend services not found, assuming none exist in project: "testproject" region: "us-test1" I0811 05:32:36.343076 40506 gce.go:1070] backend services not found, assuming none exist in project: "testproject" region: "us-test1" Disk:d1-etcd-events-minimal-gce-example-com ok FirewallRule:node-to-master-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-ipv6-minimal-gce-example-com ok FirewallRule:master-to-node-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-ipv6-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-nodes-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-minimal-gce-example-com ok FirewallRule:node-to-node-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-master-us-test1-a-minimal-gce-example-com ok Disk:d1-etcd-main-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-ipv6-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/node-minimal-gce-example-com@testproject.iam.gserviceaccount.com ok ServiceAccount:projects/testproject/serviceAccounts/control-plane-minimal-g-fu1mg6@testproject.iam.gserviceaccount.com ok FirewallRule:master-to-master-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-ipv6-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-minimal-gce-example-com ok InstanceTemplate:master-us-test1-a-minimal-do16cp-1660195956 ok InstanceTemplate:nodes-minimal-gce-example-com-1660195956 ok Subnet:us-test1-minimal-gce-example-com ok Network:minimal-gce-example-com ok --- PASS: TestLifecycleMinimalGCE (0.47s) === RUN TestLifecycleFloatingIPOpenstack I0811 05:32:36.344708 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster floatingip-openstack.k8s.local I0811 05:32:36.352824 40506 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:36.420811 40506 apply_cluster.go:467] Gossip DNS: skipping DNS validation W0811 05:32:36.568049 40506 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0811 05:32:36.568096 40506 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0811 05:32:36.582671 40506 executor.go:111] Tasks: 0 done / 98 total; 43 can run I0811 05:32:36.594586 40506 executor.go:111] Tasks: 43 done / 98 total; 45 can run I0811 05:32:36.627694 40506 executor.go:111] Tasks: 88 done / 98 total; 6 can run I0811 05:32:36.630549 40506 executor.go:111] Tasks: 94 done / 98 total; 4 can run W0811 05:32:36.632941 40506 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0811 05:32:36.633449 40506 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0811 05:32:36.633457 40506 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I0811 05:32:36.641944 40506 executor.go:111] Tasks: 98 done / 98 total; 0 can run ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:36.688048 40506 apply_cluster.go:467] Gossip DNS: skipping DNS validation W0811 05:32:36.841206 40506 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0811 05:32:36.841258 40506 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0811 05:32:36.860377 40506 executor.go:111] Tasks: 0 done / 98 total; 43 can run I0811 05:32:36.869251 40506 executor.go:111] Tasks: 43 done / 98 total; 45 can run I0811 05:32:36.877943 40506 executor.go:111] Tasks: 88 done / 98 total; 6 can run I0811 05:32:36.882155 40506 executor.go:111] Tasks: 94 done / 98 total; 4 can run W0811 05:32:36.885550 40506 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0811 05:32:36.885571 40506 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0811 05:32:36.885900 40506 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I0811 05:32:36.887941 40506 executor.go:111] Tasks: 98 done / 98 total; 0 can run Instance:e3206d30-26f2-4250-a755-cae17e76bde3 ok Router-IF:fdfc4aca-f9a0-4f6d-af3e-91dd9eb988a8 ok SecurityGroup:4d29253a-ede3-4657-bd1e-bdd76e2cbe76 ok ServerGroup:e1de5eb0-e430-42ae-a257-fae9822f7f55 ok SecurityGroup:fc6ead5b-ea91-4f1b-b594-865a00c10b71 ok Instance:8a50dfe9-1d18-4ac6-983a-d2e39fc83c74 ok Port:460a847e-01e2-4abd-b674-c941e2a65fe8 ok Subnet:fdfc4aca-f9a0-4f6d-af3e-91dd9eb988a8 ok ServerGroup:51d4eb73-3d95-4eae-9a1d-463d7db486dd ok Network:f5d7161d-0bb6-4ee3-981d-bd6cf980d53b ok Router:9dd449c9-2bbc-40e8-a8fb-a3d528be4b73 ok Instance:6334607c-0043-405c-b5d4-53542d2f6859 ok Port:9e89d210-52ac-4353-9f4b-5c59026a81cd ok SecurityGroup:6f8b15fc-a737-4936-9029-5da846603214 ok Port:669700ee-4cd9-41e1-8a37-b208916b740e ok SSHKey:kubernetes-floatingip-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok Port:40bfb709-571b-4793-ba9c-54f9cdd8b2bf ok --- PASS: TestLifecycleFloatingIPOpenstack (1.65s) === RUN TestLifecyclePrivateCalico I0811 05:32:37.996805 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:37.998003 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:37.998059 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:37.998106 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:37.998161 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:37.998203 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:37.998232 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:37.998270 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:37.998300 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:37.998361 40506 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatecalico.example.com I0811 05:32:38.032071 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0811 05:32:38.035614 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:38.035736 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:38.035809 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:38.074552 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:38.486592 40506 iamrole.go:89] ListRoles: { } I0811 05:32:38.489054 40506 executor.go:111] Tasks: 0 done / 124 total; 50 can run I0811 05:32:38.489508 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0811 05:32:38.489611 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } I0811 05:32:38.489643 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:38.489770 40506 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0811 05:32:38.489754 40506 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0811 05:32:38.489938 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0811 05:32:38.490038 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["privatecalico.example.com"] }] } I0811 05:32:38.490186 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } I0811 05:32:38.490294 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:38.490277 40506 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0811 05:32:38.490318 40506 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0811 05:32:38.490367 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:38.490437 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } I0811 05:32:38.490460 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0811 05:32:38.490537 40506 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", Tags: [{ Key: "Name", Value: "bastions.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0811 05:32:38.490709 40506 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0811 05:32:38.490771 40506 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0811 05:32:38.490823 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0811 05:32:38.490968 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0811 05:32:38.491216 40506 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0811 05:32:38.491366 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0811 05:32:38.491432 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0811 05:32:38.491495 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:38.491571 40506 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0811 05:32:38.491720 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0811 05:32:38.492012 40506 executor.go:111] Tasks: 50 done / 124 total; 28 can run I0811 05:32:38.492157 40506 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I0811 05:32:38.492192 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0811 05:32:38.492286 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I0811 05:32:38.492202 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0811 05:32:38.492443 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0811 05:32:38.492380 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I0811 05:32:38.492649 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0811 05:32:38.492733 40506 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I0811 05:32:38.492715 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0811 05:32:38.492773 40506 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0811 05:32:38.492882 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0811 05:32:38.492961 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0811 05:32:38.493237 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0811 05:32:38.493241 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0811 05:32:38.493402 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0811 05:32:38.493206 40506 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0811 05:32:38.493565 40506 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0811 05:32:38.493559 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0811 05:32:38.493776 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0811 05:32:38.493983 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] }] } I0811 05:32:38.493995 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0811 05:32:38.494134 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0811 05:32:38.494195 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0811 05:32:38.494252 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0811 05:32:38.494408 40506 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" } ] }], VpcId: "vpc-1" } I0811 05:32:38.494442 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0811 05:32:38.494603 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0811 05:32:38.494669 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:38.494732 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0811 05:32:38.494866 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0811 05:32:38.495054 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:38.494853 40506 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0811 05:32:38.495192 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:38.495316 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:38.495399 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0811 05:32:38.495511 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0811 05:32:38.495618 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0811 05:32:38.495746 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0811 05:32:38.495813 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0811 05:32:38.495868 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0811 05:32:38.495974 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0811 05:32:38.496021 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0811 05:32:38.496060 40506 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0811 05:32:38.496140 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0811 05:32:38.496187 40506 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0811 05:32:38.496246 40506 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0811 05:32:38.496318 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0811 05:32:38.496394 40506 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] } I0811 05:32:38.496498 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:32:38.496571 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0811 05:32:38.496660 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0811 05:32:38.496728 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0811 05:32:38.496795 40506 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] } I0811 05:32:38.496881 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:32:38.497554 40506 executor.go:111] Tasks: 78 done / 124 total; 40 can run W0811 05:32:38.497752 40506 api.go:53] PageSize not implemented I0811 05:32:38.497784 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-4003to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0811 05:32:38.498044 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:38.497985 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0811 05:32:38.498165 40506 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"], Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatecalico.example.com" }] } I0811 05:32:38.498245 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0811 05:32:38.498286 40506 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0811 05:32:38.498221 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I0811 05:32:38.498357 40506 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } W0811 05:32:38.498438 40506 api.go:53] PageSize not implemented I0811 05:32:38.498459 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0811 05:32:38.498480 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0811 05:32:38.498637 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0811 05:32:38.498673 40506 tags.go:74] AddTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"], Tags: [{ Key: "Name", Value: "api.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0811 05:32:38.498731 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0811 05:32:38.498763 40506 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0811 05:32:38.498812 40506 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0811 05:32:38.498775 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-1to2379-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0811 05:32:38.499035 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-4-0to0-masters.privatecalico.example.com" }] }] } I0811 05:32:38.499195 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:38.499326 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:38.499394 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:38.499462 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:38.499546 40506 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0811 05:32:38.500371 40506 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0811 05:32:38.500511 40506 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0811 05:32:38.500726 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:38.500879 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0811 05:32:38.501200 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0811 05:32:38.501455 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0811 05:32:38.501710 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } I0811 05:32:38.501922 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0811 05:32:38.501996 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0811 05:32:38.502064 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0811 05:32:38.502258 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-::/0" }] }] } I0811 05:32:38.502508 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0811 05:32:38.502775 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" }] }] } I0811 05:32:38.503138 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0811 05:32:38.504845 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-ingress-tcp-22to22-bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0811 05:32:38.505015 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I0811 05:32:38.505137 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0811 05:32:38.505278 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:38.505392 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:38.505473 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0811 05:32:38.505638 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0811 05:32:38.505815 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:38.505927 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:38.505994 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0811 05:32:38.506219 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-2382to4000-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0811 05:32:38.506395 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0811 05:32:38.506536 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0811 05:32:38.506660 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0811 05:32:38.506781 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0811 05:32:38.506891 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0811 05:32:38.507031 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0811 05:32:38.507110 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0811 05:32:38.507156 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:38.507350 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0811 05:32:38.507571 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-udp-1to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0811 05:32:38.509780 40506 executor.go:111] Tasks: 118 done / 124 total; 5 can run I0811 05:32:38.509958 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:38.510139 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:38.510213 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:38.510308 40506 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-2"] }] } I0811 05:32:38.510579 40506 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-2", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0811 05:32:38.510702 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0811 05:32:38.510751 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0811 05:32:38.510795 40506 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0811 05:32:38.510870 40506 executor.go:111] Tasks: 123 done / 124 total; 1 can run I0811 05:32:38.510933 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:38.511055 40506 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0811 05:32:38.511121 40506 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0811 05:32:38.511161 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:38.511195 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:38.511246 40506 dns.go:238] Pre-creating DNS records I0811 05:32:38.511269 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:38.518332 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0811 05:32:38.521669 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:38.521765 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:38.521831 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:38.559700 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:38.966578 40506 iamrole.go:89] ListRoles: { } I0811 05:32:38.968875 40506 executor.go:111] Tasks: 0 done / 124 total; 50 can run I0811 05:32:38.969348 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["privatecalico.example.com"] }] } I0811 05:32:38.969608 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0811 05:32:38.969741 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:38.969852 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0811 05:32:38.970027 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:38.970141 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:38.970284 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0811 05:32:38.970390 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0811 05:32:38.970421 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0811 05:32:38.970472 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:38.971881 40506 executor.go:111] Tasks: 50 done / 124 total; 28 can run I0811 05:32:38.972034 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I0811 05:32:38.972106 40506 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I0811 05:32:38.972158 40506 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I0811 05:32:38.972066 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0811 05:32:38.972255 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:38.972443 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0811 05:32:38.972657 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0811 05:32:38.972725 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0811 05:32:38.972779 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0811 05:32:38.972912 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0811 05:32:38.973928 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0811 05:32:38.974003 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0811 05:32:38.974358 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0811 05:32:38.974440 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0811 05:32:38.974665 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:38.974794 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0811 05:32:38.974990 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0811 05:32:38.975200 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I0811 05:32:38.975490 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0811 05:32:38.975701 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:38.975830 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:32:38.975907 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:38.981348 40506 executor.go:111] Tasks: 78 done / 124 total; 40 can run W0811 05:32:38.981545 40506 api.go:53] PageSize not implemented I0811 05:32:38.981616 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:38.981598 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I0811 05:32:38.981773 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0811 05:32:38.981799 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:38.981848 40506 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatecalico-example-0uch4k" } W0811 05:32:38.982015 40506 api.go:53] PageSize not implemented I0811 05:32:38.981994 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:38.982046 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I0811 05:32:38.982149 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0811 05:32:38.982119 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:38.982188 40506 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0811 05:32:38.982275 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:38.982533 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:38.982667 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:38.983043 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:38.983292 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:38.983428 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:38.983560 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:38.983642 40506 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0811 05:32:38.983688 40506 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0811 05:32:38.983737 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0811 05:32:38.985972 40506 executor.go:111] Tasks: 118 done / 124 total; 5 can run W0811 05:32:38.986091 40506 api.go:53] PageSize not implemented I0811 05:32:38.986096 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:38.986117 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I0811 05:32:38.986191 40506 dnsname.go:121] AliasTarget for "api.privatecalico.example.com" is "api-privatecalico-example-0uch4k.elb.cloudmock.com" I0811 05:32:38.986174 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:38.986280 40506 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0811 05:32:38.986225 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } W0811 05:32:38.986399 40506 api.go:53] PageSize not implemented I0811 05:32:38.986423 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I0811 05:32:38.986613 40506 executor.go:111] Tasks: 123 done / 124 total; 1 can run I0811 05:32:38.986683 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:38.986790 40506 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0811 05:32:38.990806 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0811 05:32:38.990872 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0811 05:32:38.990914 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0811 05:32:38.990952 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0811 05:32:38.990993 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0811 05:32:38.991030 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0811 05:32:38.991066 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0811 05:32:38.991099 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0811 05:32:38.991134 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0811 05:32:38.991172 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0811 05:32:38.991213 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0811 05:32:38.991250 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0811 05:32:38.991292 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0811 05:32:38.991333 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0811 05:32:38.991371 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0811 05:32:38.991409 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0811 05:32:38.991443 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0811 05:32:38.991481 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0811 05:32:38.991517 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0811 05:32:38.991556 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0811 05:32:38.991595 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0811 05:32:38.991633 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0811 05:32:38.991670 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0811 05:32:38.991708 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0811 05:32:38.991748 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:32:38.991788 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0811 05:32:38.991823 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0811 05:32:38.991865 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0811 05:32:38.991901 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0811 05:32:38.991936 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0811 05:32:38.991975 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0811 05:32:38.992014 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0811 05:32:38.992052 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0811 05:32:38.993722 40506 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0811 05:32:38.993757 40506 keypairs.go:95] DescribeKeyPairs: { } I0811 05:32:38.993802 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0811 05:32:38.993870 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0811 05:32:38.993953 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0811 05:32:38.994020 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0811 05:32:38.994070 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0811 05:32:38.994123 40506 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0811 05:32:38.994166 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0811 05:32:38.994211 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0811 05:32:38.994271 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0811 05:32:38.994325 40506 routetable.go:53] DescribeRouteTables: { } I0811 05:32:38.994358 40506 natgateway.go:117] DescribeNatGateways: { } I0811 05:32:38.994396 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0811 05:32:38.994440 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } W0811 05:32:38.994493 40506 api.go:53] PageSize not implemented I0811 05:32:38.994511 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I0811 05:32:38.994567 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0811 05:32:38.994596 40506 loadbalancers.go:34] PageSize not implemented I0811 05:32:38.994616 40506 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0811 05:32:38.994641 40506 targetgroups.go:35] PageSize not implemented I0811 05:32:38.994662 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0811 05:32:38.994718 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:38.994747 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:38.994794 40506 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0811 05:32:38.994838 40506 iamrole.go:89] ListRoles: { } I0811 05:32:38.994888 40506 internetgateways.go:101] DescribeInternetGateways: { } I0811 05:32:38.994937 40506 routetable.go:53] DescribeRouteTables: { } I0811 05:32:38.994978 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I0811 05:32:38.995018 40506 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0811 05:32:38.995055 40506 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } autoscaling-group:nodes.privatecalico.example.com ok I0811 05:32:38.995734 40506 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0811 05:32:38.995811 40506 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } autoscaling-config:lt-2 ok I0811 05:32:38.995869 40506 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatecalico-example-0uch4k" } load-balancer:api-privatecalico-example-0uch4k ok autoscaling-group:bastion.privatecalico.example.com ok I0811 05:32:38.995904 40506 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } autoscaling-group:master-us-test-1a.masters.privatecalico.example.com ok I0811 05:32:38.995936 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } keypair:key-1 ok I0811 05:32:38.995984 40506 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } I0811 05:32:38.996008 40506 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } iam-instance-profile:bastions.privatecalico.example.com ok load-balancer:bastion-privatecalico-exa-hocohm ok I0811 05:32:38.996066 40506 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0811 05:32:38.995916 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:38.996127 40506 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } route53-record:Z1AFAKE1ZON3YO/A/api.privatecalico.example.com ok iam-instance-profile:masters.privatecalico.example.com ok I0811 05:32:38.996156 40506 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0811 05:32:38.996202 40506 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } iam-instance-profile:nodes.privatecalico.example.com ok I0811 05:32:38.996036 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0811 05:32:38.996279 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:38.996307 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0811 05:32:38.996347 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0811 05:32:38.996403 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0811 05:32:38.996486 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:38.996499 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0811 05:32:38.996556 40506 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0811 05:32:38.996650 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:38.996716 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } W0811 05:32:38.996843 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:38.996862 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0811 05:32:38.996907 40506 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok autoscaling-config:lt-3 ok autoscaling-config:lt-1 ok I0811 05:32:38.997015 40506 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0811 05:32:38.997074 40506 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0811 05:32:38.997182 40506 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I0811 05:32:38.997195 40506 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatecalico.example.com" } I0811 05:32:38.997268 40506 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatecalico.example.com" } I0811 05:32:38.997254 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0811 05:32:38.997321 40506 iamrole.go:180] ListAttachedRolePolicies: bastions.privatecalico.example.com I0811 05:32:38.997346 40506 iamrole.go:180] ListAttachedRolePolicies: nodes.privatecalico.example.com I0811 05:32:38.997379 40506 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0811 05:32:38.997430 40506 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatecalico.example.com" } iam-role:nodes.privatecalico.example.com ok I0811 05:32:38.997338 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 } ] } I0811 05:32:38.997471 40506 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } W0811 05:32:38.997494 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:38.997502 40506 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatecalico.example.com" } iam-role:bastions.privatecalico.example.com ok I0811 05:32:38.997516 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0811 05:32:38.997533 40506 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatecalico.example.com" } I0811 05:32:38.997555 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:38.997586 40506 iamrole.go:180] ListAttachedRolePolicies: masters.privatecalico.example.com I0811 05:32:38.997604 40506 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0811 05:32:38.997633 40506 iamrole.go:137] DeleteRole: { RoleName: "masters.privatecalico.example.com" } iam-role:masters.privatecalico.example.com ok I0811 05:32:38.997611 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }] } W0811 05:32:38.997713 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:38.997728 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0811 05:32:38.997765 40506 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0811 05:32:38.997834 40506 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0811 05:32:38.997934 40506 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0811 05:32:38.997999 40506 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0811 05:32:38.998082 40506 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0811 05:32:38.998166 40506 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0811 05:32:38.998578 40506 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateCalico (1.00s) === RUN TestLifecyclePrivateKopeio I0811 05:32:38.998927 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:38.999244 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:38.999291 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:38.999330 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:38.999379 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:38.999423 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:38.999452 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:38.999487 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:38.999519 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:38.999575 40506 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatekopeio.example.com I0811 05:32:39.028522 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:39.031638 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:39.031731 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:39.031799 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:39.069034 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:39.179173 40506 iamrole.go:89] ListRoles: { } I0811 05:32:39.181400 40506 executor.go:111] Tasks: 0 done / 123 total; 47 can run I0811 05:32:39.181706 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.181933 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } I0811 05:32:39.181978 40506 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0811 05:32:39.182043 40506 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0811 05:32:39.182086 40506 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0811 05:32:39.182130 40506 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0811 05:32:39.182170 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0811 05:32:39.182230 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:39.182225 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } I0811 05:32:39.182299 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:39.182335 40506 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] } I0811 05:32:39.182325 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:39.182510 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } I0811 05:32:39.182402 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.182585 40506 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0811 05:32:39.182675 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:39.182855 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] }] } I0811 05:32:39.183091 40506 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0811 05:32:39.183231 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0811 05:32:39.183285 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.183459 40506 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0811 05:32:39.183567 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0811 05:32:39.183612 40506 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0811 05:32:39.183735 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0811 05:32:39.183781 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0811 05:32:39.184921 40506 executor.go:111] Tasks: 47 done / 123 total; 31 can run I0811 05:32:39.185104 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0811 05:32:39.185204 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0811 05:32:39.185239 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0811 05:32:39.185376 40506 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0811 05:32:39.185409 40506 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I0811 05:32:39.185464 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0811 05:32:39.185397 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.185582 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0811 05:32:39.185619 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:39.185840 40506 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0811 05:32:39.185892 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0811 05:32:39.185939 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0811 05:32:39.186098 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0811 05:32:39.186206 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0811 05:32:39.186259 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:39.186298 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.186402 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.186502 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "masters.privatekopeio.example.com" }] }], VpcId: "vpc-1" } I0811 05:32:39.186526 40506 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0811 05:32:39.186594 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.186619 40506 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0811 05:32:39.186688 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0811 05:32:39.186651 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0811 05:32:39.186796 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.186811 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0811 05:32:39.186890 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I0811 05:32:39.186990 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.187082 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:39.187037 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.8.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0811 05:32:39.187151 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0811 05:32:39.187234 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.187228 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0811 05:32:39.187323 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.187400 40506 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0811 05:32:39.187477 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0811 05:32:39.187534 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0811 05:32:39.187600 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:39.187639 40506 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0811 05:32:39.188237 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:32:39.189009 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0811 05:32:39.189249 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0811 05:32:39.189462 40506 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0811 05:32:39.189542 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0811 05:32:39.189619 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0811 05:32:39.189203 40506 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0811 05:32:39.189757 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1b"] }] } I0811 05:32:39.189921 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:32:39.190031 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0811 05:32:39.189977 40506 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I0811 05:32:39.190150 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0811 05:32:39.190162 40506 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" } ] }], VpcId: "vpc-1" } I0811 05:32:39.190264 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0811 05:32:39.190413 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0811 05:32:39.190513 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0811 05:32:39.190596 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0811 05:32:39.190690 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0811 05:32:39.190832 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0811 05:32:39.190974 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0811 05:32:39.191101 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0811 05:32:39.191253 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0811 05:32:39.191471 40506 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0811 05:32:39.191737 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0811 05:32:39.192025 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0811 05:32:39.193906 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0811 05:32:39.194035 40506 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" } ] } I0811 05:32:39.194173 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0811 05:32:39.194250 40506 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" }, { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] } I0811 05:32:39.194561 40506 executor.go:111] Tasks: 78 done / 123 total; 39 can run I0811 05:32:39.194740 40506 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } W0811 05:32:39.194867 40506 api.go:53] PageSize not implemented I0811 05:32:39.194890 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0811 05:32:39.195031 40506 api.go:53] PageSize not implemented I0811 05:32:39.195020 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:39.195193 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0811 05:32:39.195234 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0811 05:32:39.195306 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0811 05:32:39.195343 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-3" } I0811 05:32:39.195424 40506 tags.go:74] AddTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"], Tags: [{ Key: "Name", Value: "api.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0811 05:32:39.195501 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0811 05:32:39.195537 40506 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0811 05:32:39.195599 40506 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0811 05:32:39.195537 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0811 05:32:39.195695 40506 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"], Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0811 05:32:39.195759 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0811 05:32:39.195793 40506 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0811 05:32:39.195840 40506 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0811 05:32:39.195838 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I0811 05:32:39.195987 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I0811 05:32:39.196106 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:39.196193 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0811 05:32:39.196243 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0811 05:32:39.196401 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0811 05:32:39.196554 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:39.196639 40506 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0811 05:32:39.196734 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0811 05:32:39.196797 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0811 05:32:39.196842 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0811 05:32:39.196925 40506 tags.go:42] CreateTags { Resources: ["nat-a2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0811 05:32:39.196977 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0811 05:32:39.197024 40506 tags.go:42] CreateTags { Resources: ["subnet-4"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0811 05:32:39.197066 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0811 05:32:39.197107 40506 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0811 05:32:39.197152 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0811 05:32:39.197306 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:39.197453 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-4"] }] } I0811 05:32:39.197514 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0811 05:32:39.197589 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:39.197693 40506 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0811 05:32:39.197753 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0811 05:32:39.197931 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatekopeio.example.com" }] }] } I0811 05:32:39.198128 40506 tags.go:42] CreateTags { Resources: ["nat-b2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0811 05:32:39.198195 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-1to2379-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0811 05:32:39.198427 40506 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0811 05:32:39.198504 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-2382to4000-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0811 05:32:39.198738 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-udp-1to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0811 05:32:39.198914 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } I0811 05:32:39.199033 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0811 05:32:39.199164 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0811 05:32:39.199264 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:39.199343 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0811 05:32:39.199468 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0811 05:32:39.199609 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0811 05:32:39.199740 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:39.199817 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0811 05:32:39.199940 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-ingress-tcp-22to22-bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0811 05:32:39.200074 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-4" } I0811 05:32:39.200110 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0811 05:32:39.200233 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0811 05:32:39.200278 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:39.200327 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0811 05:32:39.200450 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-4003to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0811 05:32:39.200580 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0811 05:32:39.200703 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0811 05:32:39.200751 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:39.200798 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:39.200940 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-masters.privatekopeio.example.com" }] }] } I0811 05:32:39.201306 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-2" } I0811 05:32:39.201423 40506 tags.go:42] CreateTags { Resources: ["subnet-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0811 05:32:39.201533 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0811 05:32:39.201645 40506 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0811 05:32:39.203351 40506 executor.go:111] Tasks: 117 done / 123 total; 6 can run I0811 05:32:39.203453 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:39.203477 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0811 05:32:39.203600 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:39.203600 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:39.203771 40506 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-a2345678", RouteTableId: "rtb-2" } I0811 05:32:39.203829 40506 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-b2345678", RouteTableId: "rtb-3" } I0811 05:32:39.203902 40506 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0811 05:32:39.203943 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:39.203975 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:39.204024 40506 dns.go:238] Pre-creating DNS records I0811 05:32:39.204049 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:39.210490 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:39.213574 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:39.213671 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:39.213739 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:39.250788 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:39.368285 40506 iamrole.go:89] ListRoles: { } I0811 05:32:39.370539 40506 executor.go:111] Tasks: 0 done / 123 total; 47 can run I0811 05:32:39.370820 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:39.370776 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:39.370911 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:39.371060 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.371318 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.371505 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0811 05:32:39.371578 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0811 05:32:39.371648 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.371866 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.372017 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:39.373996 40506 executor.go:111] Tasks: 47 done / 123 total; 31 can run I0811 05:32:39.374195 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:39.374348 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0811 05:32:39.374290 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0811 05:32:39.374524 40506 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I0811 05:32:39.374506 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.374673 40506 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0811 05:32:39.374680 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:39.374781 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0811 05:32:39.374840 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.374972 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.375119 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0811 05:32:39.375162 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I0811 05:32:39.375333 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0811 05:32:39.375383 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.375543 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.375705 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0811 05:32:39.375753 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.375879 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:32:39.375929 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0811 05:32:39.376160 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:39.376226 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0811 05:32:39.376374 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0811 05:32:39.376424 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0811 05:32:39.376580 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:39.376648 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.376781 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.376902 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0811 05:32:39.379315 40506 executor.go:111] Tasks: 78 done / 123 total; 39 can run I0811 05:32:39.379583 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } W0811 05:32:39.379693 40506 api.go:53] PageSize not implemented I0811 05:32:39.379726 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:39.379729 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8","bastion-privatekopeio-exa-d8ef8e"] } I0811 05:32:39.379836 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0811 05:32:39.379831 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0811 05:32:39.379882 40506 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } W0811 05:32:39.379973 40506 api.go:53] PageSize not implemented I0811 05:32:39.379996 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8","bastion-privatekopeio-exa-d8ef8e"] } I0811 05:32:39.379934 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:39.380073 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0811 05:32:39.380107 40506 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0811 05:32:39.380116 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:39.380327 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:39.380508 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:39.380659 40506 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0811 05:32:39.380760 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:39.381002 40506 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0811 05:32:39.381201 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:39.381333 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:39.381433 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:39.381487 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:39.383792 40506 executor.go:111] Tasks: 117 done / 123 total; 6 can run W0811 05:32:39.383912 40506 api.go:53] PageSize not implemented I0811 05:32:39.383900 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0811 05:32:39.383942 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0811 05:32:39.384041 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0811 05:32:39.384136 40506 api.go:53] PageSize not implemented I0811 05:32:39.384055 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:39.384195 40506 dnsname.go:121] AliasTarget for "api.privatekopeio.example.com" is "api-privatekopeio-example-tl2bv8.elb.cloudmock.com" I0811 05:32:39.384168 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0811 05:32:39.384381 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0811 05:32:39.384457 40506 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0811 05:32:39.388040 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0811 05:32:39.388108 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0811 05:32:39.388152 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0811 05:32:39.388190 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0811 05:32:39.388233 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0811 05:32:39.388272 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0811 05:32:39.388308 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0811 05:32:39.388347 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0811 05:32:39.388388 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0811 05:32:39.388431 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0811 05:32:39.388476 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0811 05:32:39.388519 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0811 05:32:39.388567 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0811 05:32:39.388608 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0811 05:32:39.388647 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0811 05:32:39.388689 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0811 05:32:39.388726 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0811 05:32:39.388767 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0811 05:32:39.388810 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0811 05:32:39.388851 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0811 05:32:39.388893 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0811 05:32:39.388934 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0811 05:32:39.388973 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0811 05:32:39.389033 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:32:39.389081 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0811 05:32:39.389120 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0811 05:32:39.389162 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0811 05:32:39.389204 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0811 05:32:39.389246 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0811 05:32:39.389282 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0811 05:32:39.389318 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0811 05:32:39.389358 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0811 05:32:39.389398 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0811 05:32:39.389437 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0811 05:32:39.391159 40506 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0811 05:32:39.391192 40506 keypairs.go:95] DescribeKeyPairs: { } I0811 05:32:39.391231 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.391297 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0811 05:32:39.391377 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.391441 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.391490 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.391548 40506 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.391592 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.391644 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0811 05:32:39.391712 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.391783 40506 routetable.go:53] DescribeRouteTables: { } I0811 05:32:39.391819 40506 natgateway.go:117] DescribeNatGateways: { } I0811 05:32:39.391859 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0811 05:32:39.391906 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } W0811 05:32:39.391965 40506 api.go:53] PageSize not implemented I0811 05:32:39.391984 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8","bastion-privatekopeio-exa-d8ef8e"] } I0811 05:32:39.392039 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0811 05:32:39.392068 40506 loadbalancers.go:34] PageSize not implemented I0811 05:32:39.392087 40506 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0811 05:32:39.392112 40506 targetgroups.go:35] PageSize not implemented I0811 05:32:39.392131 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0811 05:32:39.392182 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:39.392210 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:39.392261 40506 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0811 05:32:39.392301 40506 iamrole.go:89] ListRoles: { } I0811 05:32:39.392370 40506 internetgateways.go:101] DescribeInternetGateways: { } I0811 05:32:39.392426 40506 routetable.go:53] DescribeRouteTables: { } I0811 05:32:39.392478 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2","rtb-3"] } I0811 05:32:39.392526 40506 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0811 05:32:39.392560 40506 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0811 05:32:39.393240 40506 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0811 05:32:39.393314 40506 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } autoscaling-group:bastion.privatekopeio.example.com ok autoscaling-config:lt-2 ok I0811 05:32:39.393384 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } internet-gateway:igw-2 ok I0811 05:32:39.393466 40506 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } load-balancer:api-privatekopeio-example-tl2bv8 ok autoscaling-config:lt-1 ok I0811 05:32:39.393386 40506 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0811 05:32:39.393479 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } autoscaling-group:master-us-test-1a.masters.privatekopeio.example.com ok W0811 05:32:39.393590 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:39.393393 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:39.393553 40506 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.privatekopeio.example.com. ok I0811 05:32:39.393610 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0811 05:32:39.393630 40506 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } iam-instance-profile:masters.privatekopeio.example.com ok I0811 05:32:39.393646 40506 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } autoscaling-group:nodes.privatekopeio.example.com ok I0811 05:32:39.393667 40506 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } load-balancer:bastion-privatekopeio-exa-d8ef8e ok iam-instance-profile:nodes.privatekopeio.example.com ok I0811 05:32:39.393645 40506 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0811 05:32:39.393710 40506 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0811 05:32:39.393760 40506 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } iam-instance-profile:bastions.privatekopeio.example.com ok I0811 05:32:39.393770 40506 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0811 05:32:39.393859 40506 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0811 05:32:39.393916 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0811 05:32:39.393979 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } W0811 05:32:39.394117 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:39.394141 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0811 05:32:39.394177 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:39.394234 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0811 05:32:39.394435 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:39.394454 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-3 ok I0811 05:32:39.394595 40506 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-4" } subnet:subnet-4 ok I0811 05:32:39.394637 40506 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatekopeio.example.com" } I0811 05:32:39.394724 40506 iamrole.go:180] ListAttachedRolePolicies: masters.privatekopeio.example.com I0811 05:32:39.394745 40506 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0811 05:32:39.394665 40506 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I0811 05:32:39.394784 40506 iamrole.go:137] DeleteRole: { RoleName: "masters.privatekopeio.example.com" } iam-role:masters.privatekopeio.example.com ok I0811 05:32:39.394841 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:39.394897 40506 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatekopeio.example.com" } I0811 05:32:39.394970 40506 iamrole.go:180] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0811 05:32:39.394989 40506 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0811 05:32:39.394928 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } I0811 05:32:39.395023 40506 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatekopeio.example.com" } iam-role:nodes.privatekopeio.example.com ok W0811 05:32:39.395043 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:39.395059 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } I0811 05:32:39.395055 40506 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatekopeio.example.com" } security-group:sg-1 ok I0811 05:32:39.395099 40506 iamrole.go:180] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0811 05:32:39.395111 40506 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0811 05:32:39.395141 40506 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatekopeio.example.com" } iam-role:bastions.privatekopeio.example.com ok I0811 05:32:39.395133 40506 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0811 05:32:39.395243 40506 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0811 05:32:39.395295 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0811 05:32:39.395349 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0811 05:32:39.395442 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:39.395456 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0811 05:32:39.395552 40506 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0811 05:32:39.395617 40506 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0811 05:32:39.395675 40506 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-3" } route-table:rtb-3 ok I0811 05:32:39.395784 40506 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0811 05:32:39.395867 40506 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0811 05:32:39.396154 40506 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateKopeio (0.40s) === RUN TestLifecycleIPv6 I0811 05:32:39.396521 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:39.396810 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:39.396859 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:39.396902 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:39.396954 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:39.396999 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:39.397031 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:39.397073 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:39.397102 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:39.397157 40506 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal-ipv6.example.com I0811 05:32:39.428500 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:39.432064 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:39.432163 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:39.469238 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:39.634641 40506 iamrole.go:89] ListRoles: { } I0811 05:32:39.636206 40506 executor.go:111] Tasks: 0 done / 104 total; 45 can run I0811 05:32:39.636596 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0811 05:32:39.636713 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } I0811 05:32:39.638156 40506 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0811 05:32:39.638213 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:39.638287 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } I0811 05:32:39.638397 40506 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0811 05:32:39.636812 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0811 05:32:39.639962 40506 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0811 05:32:39.640517 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0811 05:32:39.640699 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0811 05:32:39.641140 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0811 05:32:39.642806 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:39.642860 40506 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0811 05:32:39.643058 40506 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0811 05:32:39.643048 40506 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0811 05:32:39.643183 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0811 05:32:39.643238 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0811 05:32:39.643362 40506 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0811 05:32:39.643407 40506 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0811 05:32:39.643441 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0811 05:32:39.643493 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:39.643535 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:39.643713 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0811 05:32:39.644343 40506 executor.go:111] Tasks: 45 done / 104 total; 21 can run I0811 05:32:39.644519 40506 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0811 05:32:39.644624 40506 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0811 05:32:39.644572 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:39.644721 40506 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-minimal-ipv6-example--bne5ih", Port: 443, Protocol: "TCP", Tags: [{ Key: "Name", Value: "tcp-minimal-ipv6-example--bne5ih" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0811 05:32:39.644782 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0811 05:32:39.644797 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:39.644940 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0811 05:32:39.645020 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0811 05:32:39.644937 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0811 05:32:39.645098 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0811 05:32:39.645185 40506 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0811 05:32:39.645235 40506 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0811 05:32:39.645258 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0811 05:32:39.645479 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0811 05:32:39.645606 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0811 05:32:39.645621 40506 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0811 05:32:39.645697 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0811 05:32:39.645911 40506 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0811 05:32:39.646005 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0811 05:32:39.646131 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0811 05:32:39.646220 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0811 05:32:39.646325 40506 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0811 05:32:39.646364 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:39.646412 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0811 05:32:39.646462 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0811 05:32:39.646591 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0811 05:32:39.646645 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:39.646691 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0811 05:32:39.646740 40506 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0811 05:32:39.646860 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0811 05:32:39.646994 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0811 05:32:39.647065 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0811 05:32:39.647111 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public-us-test-1a"] }] } I0811 05:32:39.647247 40506 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" } ] }], VpcId: "vpc-1" } I0811 05:32:39.647335 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0811 05:32:39.647380 40506 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" }, { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] } I0811 05:32:39.646955 40506 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0811 05:32:39.649991 40506 executor.go:111] Tasks: 66 done / 104 total; 30 can run I0811 05:32:39.650139 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:39.650243 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:39.650412 40506 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0811 05:32:39.650476 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:39.650608 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:39.650766 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0811 05:32:39.650957 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0811 05:32:39.651196 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" }] }] } I0811 05:32:39.651358 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:39.651503 40506 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0811 05:32:39.651563 40506 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0811 05:32:39.651622 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0811 05:32:39.651728 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:39.651844 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0811 05:32:39.652067 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" }] }] } I0811 05:32:39.652299 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0811 05:32:39.652569 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] }] } I0811 05:32:39.652733 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:39.652837 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", Ipv6CidrBlock: "2001:db8:0:111::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" } ] }], VpcId: "vpc-1" } I0811 05:32:39.652816 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-udp-1to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0811 05:32:39.652892 40506 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0811 05:32:39.653004 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-4003to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0811 05:32:39.653217 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" }] }] } I0811 05:32:39.653386 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0811 05:32:39.653489 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0811 05:32:39.653631 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:39.653723 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0811 05:32:39.653818 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-::/0" }] }] } I0811 05:32:39.653961 40506 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0811 05:32:39.654007 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0811 05:32:39.654138 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0811 05:32:39.654290 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-1to2379-masters.minimal-ipv6.example.com" }] }] } I0811 05:32:39.654449 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-2382to4000-masters.minimal-ipv6.example.com" }] }] } I0811 05:32:39.654602 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0811 05:32:39.654763 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0811 05:32:39.654899 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:39.654952 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:32:39.655005 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:32:39.657781 40506 executor.go:111] Tasks: 96 done / 104 total; 5 can run I0811 05:32:39.657877 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:39.657971 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:39.658078 40506 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:32:39.658124 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } I0811 05:32:39.658174 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0811 05:32:39.658246 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } W0811 05:32:39.658217 40506 loadbalancers.go:34] PageSize not implemented I0811 05:32:39.658216 40506 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0811 05:32:39.658414 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0811 05:32:39.658471 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:32:39.658430 40506 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "dualstack", Name: "api-minimal-ipv6-example--jhj9te", SubnetMappings: [{ SubnetId: "subnet-1" }], Tags: [{ Key: "Name", Value: "api.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], Type: "network" } I0811 05:32:39.658516 40506 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0811 05:32:39.658564 40506 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1", Port: 443, Protocol: "TCP" } I0811 05:32:39.658661 40506 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "false" },{ Key: "access_logs.s3.enabled", Value: "false" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0811 05:32:39.658774 40506 executor.go:111] Tasks: 101 done / 104 total; 3 can run I0811 05:32:39.658864 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:39.658927 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:39.659034 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:39.659105 40506 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "64:ff9b::/96", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0811 05:32:39.659138 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:39.659289 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:39.659390 40506 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0811 05:32:39.659430 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:39.659465 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:39.659519 40506 dns.go:238] Pre-creating DNS records I0811 05:32:39.659545 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:39.665352 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:39.668590 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:39.668686 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:39.705620 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:39.879052 40506 iamrole.go:89] ListRoles: { } I0811 05:32:39.880997 40506 executor.go:111] Tasks: 0 done / 104 total; 45 can run I0811 05:32:39.881335 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0811 05:32:39.881353 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:39.881498 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0811 05:32:39.881536 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0811 05:32:39.881604 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:39.881677 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:39.881779 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:39.881959 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0811 05:32:39.882144 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0811 05:32:39.882281 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0811 05:32:39.884227 40506 executor.go:111] Tasks: 45 done / 104 total; 21 can run I0811 05:32:39.884379 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:39.884472 40506 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0811 05:32:39.884524 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0811 05:32:39.884701 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0811 05:32:39.884632 40506 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0811 05:32:39.884694 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0811 05:32:39.884776 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0811 05:32:39.884873 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0811 05:32:39.885057 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0811 05:32:39.885118 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0811 05:32:39.885259 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:39.885315 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0811 05:32:39.885534 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:39.885593 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:39.885716 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0811 05:32:39.889610 40506 executor.go:111] Tasks: 66 done / 104 total; 30 can run I0811 05:32:39.889767 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:39.889930 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:39.890129 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0811 05:32:39.890325 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:32:39.890587 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:39.890742 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:39.890871 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:39.890993 40506 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0811 05:32:39.891037 40506 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0811 05:32:39.891073 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0811 05:32:39.891188 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:39.891312 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:39.891422 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:39.891595 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:39.894097 40506 executor.go:111] Tasks: 96 done / 104 total; 5 can run I0811 05:32:39.894213 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:39.894314 40506 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0811 05:32:39.894284 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0811 05:32:39.894371 40506 loadbalancers.go:34] PageSize not implemented I0811 05:32:39.894390 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0811 05:32:39.894359 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:39.894436 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0811 05:32:39.894475 40506 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0811 05:32:39.894520 40506 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0811 05:32:39.894552 40506 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0811 05:32:39.894667 40506 executor.go:111] Tasks: 101 done / 104 total; 3 can run I0811 05:32:39.894742 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:39.894819 40506 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0811 05:32:39.894788 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:39.894845 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0811 05:32:39.894943 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0811 05:32:39.894852 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:39.895029 40506 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0811 05:32:39.895059 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0811 05:32:39.895089 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0811 05:32:39.895186 40506 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0811 05:32:39.898208 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0811 05:32:39.898274 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0811 05:32:39.898312 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0811 05:32:39.898349 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0811 05:32:39.898389 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0811 05:32:39.898428 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0811 05:32:39.898464 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0811 05:32:39.898499 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0811 05:32:39.898533 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0811 05:32:39.898570 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0811 05:32:39.898604 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0811 05:32:39.898644 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0811 05:32:39.898689 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0811 05:32:39.898726 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0811 05:32:39.898760 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0811 05:32:39.898794 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0811 05:32:39.898832 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0811 05:32:39.898866 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0811 05:32:39.898903 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0811 05:32:39.898940 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0811 05:32:39.898979 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0811 05:32:39.899034 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:32:39.899079 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0811 05:32:39.899114 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0811 05:32:39.899148 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0811 05:32:39.899181 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0811 05:32:39.899220 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0811 05:32:39.899258 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0811 05:32:39.899298 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0811 05:32:39.900844 40506 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0811 05:32:39.900878 40506 keypairs.go:95] DescribeKeyPairs: { } I0811 05:32:39.900920 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0811 05:32:39.900976 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0811 05:32:39.901041 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0811 05:32:39.901103 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0811 05:32:39.901159 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0811 05:32:39.901214 40506 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0811 05:32:39.901259 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0811 05:32:39.901305 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0811 05:32:39.901370 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0811 05:32:39.901418 40506 routetable.go:53] DescribeRouteTables: { } I0811 05:32:39.901447 40506 natgateway.go:117] DescribeNatGateways: { } I0811 05:32:39.901488 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0811 05:32:39.901531 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } W0811 05:32:39.901585 40506 api.go:53] PageSize not implemented I0811 05:32:39.901605 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0811 05:32:39.901632 40506 loadbalancers.go:34] PageSize not implemented I0811 05:32:39.901646 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0811 05:32:39.901687 40506 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0811 05:32:39.901712 40506 targetgroups.go:35] PageSize not implemented I0811 05:32:39.901730 40506 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" ] } I0811 05:32:39.901792 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:39.901819 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:39.901863 40506 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0811 05:32:39.901898 40506 iamrole.go:89] ListRoles: { } I0811 05:32:39.901943 40506 internetgateways.go:101] DescribeInternetGateways: { } I0811 05:32:39.901993 40506 routetable.go:53] DescribeRouteTables: { } I0811 05:32:39.902037 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0811 05:32:39.902075 40506 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0811 05:32:39.902111 40506 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0811 05:32:39.902682 40506 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0811 05:32:39.902754 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0811 05:32:39.902796 40506 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } autoscaling-group:nodes.minimal-ipv6.example.com ok I0811 05:32:39.902855 40506 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } iam-instance-profile:masters.minimal-ipv6.example.com ok I0811 05:32:39.902814 40506 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } autoscaling-group:master-us-test-1a.masters.minimal-ipv6.example.com ok I0811 05:32:39.902923 40506 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4 ok I0811 05:32:39.902939 40506 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1 ok I0811 05:32:39.902999 40506 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } iam-instance-profile:nodes.minimal-ipv6.example.com ok I0811 05:32:39.903000 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [ { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }, { Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } } ] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/AAAA/api.minimal-ipv6.example.com ok I0811 05:32:39.902842 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 } ] } W0811 05:32:39.903259 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:39.903287 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok autoscaling-config:lt-1 ok I0811 05:32:39.903366 40506 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0811 05:32:39.903444 40506 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0811 05:32:39.903497 40506 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0811 05:32:39.903565 40506 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0811 05:32:39.903610 40506 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok autoscaling-config:lt-2 ok I0811 05:32:39.903693 40506 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0811 05:32:39.903755 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:39.903809 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 } ] } W0811 05:32:39.903977 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:39.904003 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0811 05:32:39.904048 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:39.904113 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0811 05:32:39.904237 40506 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal-ipv6.example.com" } I0811 05:32:39.904256 40506 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } I0811 05:32:39.904317 40506 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com subnet:subnet-1 ok I0811 05:32:39.904336 40506 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0811 05:32:39.904370 40506 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal-ipv6.example.com" } iam-role:nodes.minimal-ipv6.example.com ok I0811 05:32:39.904351 40506 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I0811 05:32:39.904399 40506 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal-ipv6.example.com" } I0811 05:32:39.904458 40506 iamrole.go:180] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0811 05:32:39.904477 40506 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0811 05:32:39.904515 40506 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal-ipv6.example.com" } iam-role:masters.minimal-ipv6.example.com ok I0811 05:32:39.904600 40506 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0811 05:32:39.904689 40506 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0811 05:32:39.904779 40506 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0811 05:32:39.905055 40506 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleIPv6 (0.51s) === RUN TestLifecycleSharedVPC I0811 05:32:39.905325 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:39.905597 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:39.905644 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:39.905728 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:39.905788 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:39.905830 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:39.905862 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:39.905900 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:39.905929 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:39.905986 40506 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedvpc.example.com I0811 05:32:39.935777 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:39.938565 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:39.938663 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:39.975288 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:40.070968 40506 iamrole.go:89] ListRoles: { } I0811 05:32:40.072219 40506 executor.go:111] Tasks: 0 done / 82 total; 42 can run I0811 05:32:40.074309 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0811 05:32:40.074472 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0811 05:32:40.074475 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } I0811 05:32:40.074535 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0811 05:32:40.075073 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] }] } I0811 05:32:40.075341 40506 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0811 05:32:40.075528 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0811 05:32:40.074574 40506 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] } I0811 05:32:40.075767 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0811 05:32:40.075999 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:40.076077 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:40.076700 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } I0811 05:32:40.077516 40506 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0811 05:32:40.076331 40506 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0811 05:32:40.076351 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:40.077667 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0811 05:32:40.078333 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0811 05:32:40.079783 40506 executor.go:111] Tasks: 42 done / 82 total; 17 can run I0811 05:32:40.079973 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0811 05:32:40.080039 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0811 05:32:40.080127 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0811 05:32:40.080247 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0811 05:32:40.080205 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0811 05:32:40.080318 40506 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I0811 05:32:40.080431 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.sharedvpc.example.com" }] }], VpcId: "vpc-12345678" } I0811 05:32:40.080554 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0811 05:32:40.080463 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-12345678" } I0811 05:32:40.080616 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:40.080679 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0811 05:32:40.080787 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0811 05:32:40.080907 40506 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-12345678" } I0811 05:32:40.080924 40506 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0811 05:32:40.080875 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0811 05:32:40.081146 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0811 05:32:40.081264 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0811 05:32:40.081376 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:32:40.081432 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:32:40.081480 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0811 05:32:40.081526 40506 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "sharedvpc.example.com" } ] } I0811 05:32:40.081605 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0811 05:32:40.081660 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:40.082473 40506 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0811 05:32:40.085203 40506 executor.go:111] Tasks: 59 done / 82 total; 21 can run I0811 05:32:40.085375 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0811 05:32:40.085539 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-4003to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0811 05:32:40.085805 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0811 05:32:40.085967 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0811 05:32:40.086193 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:40.086309 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:40.086496 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0811 05:32:40.086685 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0811 05:32:40.086905 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0811 05:32:40.087059 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:40.087141 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:40.087206 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:40.087279 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-1to2379-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0811 05:32:40.087520 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0811 05:32:40.087852 40506 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0811 05:32:40.087954 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" }] }] } I0811 05:32:40.088225 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0811 05:32:40.088396 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:40.088524 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:40.088584 40506 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0811 05:32:40.088670 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-2382to4000-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0811 05:32:40.088861 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0811 05:32:40.089029 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0811 05:32:40.089099 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0811 05:32:40.089144 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-udp-1to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0811 05:32:40.089682 40506 executor.go:111] Tasks: 80 done / 82 total; 2 can run I0811 05:32:40.089979 40506 executor.go:111] Tasks: 82 done / 82 total; 0 can run I0811 05:32:40.090033 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:40.090086 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:40.090144 40506 dns.go:238] Pre-creating DNS records I0811 05:32:40.090176 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:40.096296 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:40.099074 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:40.099172 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:40.137320 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:40.237972 40506 iamrole.go:89] ListRoles: { } I0811 05:32:40.239588 40506 executor.go:111] Tasks: 0 done / 82 total; 42 can run I0811 05:32:40.240215 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0811 05:32:40.240362 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:40.240377 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:40.240453 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:40.240633 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0811 05:32:40.240789 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0811 05:32:40.240866 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0811 05:32:40.240896 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0811 05:32:40.245574 40506 executor.go:111] Tasks: 42 done / 82 total; 17 can run I0811 05:32:40.246328 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0811 05:32:40.246395 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0811 05:32:40.246499 40506 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I0811 05:32:40.246566 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0811 05:32:40.246824 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:40.246905 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:32:40.246986 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0811 05:32:40.247184 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:40.247275 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0811 05:32:40.247495 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0811 05:32:40.251588 40506 executor.go:111] Tasks: 59 done / 82 total; 21 can run I0811 05:32:40.251814 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:40.251958 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:40.252135 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:40.252364 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:40.252556 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:40.252892 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:40.253036 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:40.254689 40506 executor.go:111] Tasks: 80 done / 82 total; 2 can run I0811 05:32:40.254983 40506 executor.go:111] Tasks: 82 done / 82 total; 0 can run I0811 05:32:40.258192 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0811 05:32:40.258258 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0811 05:32:40.258296 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0811 05:32:40.258328 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0811 05:32:40.258362 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0811 05:32:40.258396 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0811 05:32:40.258429 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0811 05:32:40.258467 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0811 05:32:40.258519 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0811 05:32:40.258592 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0811 05:32:40.258651 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0811 05:32:40.258716 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0811 05:32:40.258783 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0811 05:32:40.258839 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0811 05:32:40.258908 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0811 05:32:40.258972 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:32:40.259070 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0811 05:32:40.259130 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0811 05:32:40.259180 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0811 05:32:40.259216 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0811 05:32:40.259254 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0811 05:32:40.259291 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0811 05:32:40.260711 40506 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0811 05:32:40.260740 40506 keypairs.go:95] DescribeKeyPairs: { } I0811 05:32:40.260781 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0811 05:32:40.260834 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0811 05:32:40.260892 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0811 05:32:40.260953 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0811 05:32:40.260997 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0811 05:32:40.261041 40506 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0811 05:32:40.261081 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0811 05:32:40.261120 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0811 05:32:40.261166 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0811 05:32:40.261221 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0811 05:32:40.261256 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } W0811 05:32:40.261305 40506 api.go:53] PageSize not implemented I0811 05:32:40.261327 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0811 05:32:40.261356 40506 loadbalancers.go:34] PageSize not implemented I0811 05:32:40.261378 40506 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0811 05:32:40.261403 40506 targetgroups.go:35] PageSize not implemented I0811 05:32:40.261421 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0811 05:32:40.261474 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:40.261504 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:40.261549 40506 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0811 05:32:40.261589 40506 iamrole.go:89] ListRoles: { } I0811 05:32:40.261632 40506 internetgateways.go:101] DescribeInternetGateways: { } I0811 05:32:40.261672 40506 routetable.go:53] DescribeRouteTables: { } I0811 05:32:40.261703 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-config:lt-2 ok I0811 05:32:40.262159 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-group:nodes.sharedvpc.example.com ok I0811 05:32:40.262208 40506 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0811 05:32:40.262292 40506 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } iam-instance-profile:masters.sharedvpc.example.com ok I0811 05:32:40.262221 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0811 05:32:40.262453 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:40.262426 40506 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0811 05:32:40.262473 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0811 05:32:40.262480 40506 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } autoscaling-group:master-us-test-1a.masters.sharedvpc.example.com ok I0811 05:32:40.262303 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } iam-instance-profile:nodes.sharedvpc.example.com ok route53-record:Z1AFAKE1ZON3YO/A/api.sharedvpc.example.com. ok I0811 05:32:40.262546 40506 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0811 05:32:40.262669 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:40.262760 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0811 05:32:40.262918 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:40.262945 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-1 ok I0811 05:32:40.263061 40506 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0811 05:32:40.263164 40506 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0811 05:32:40.263293 40506 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0811 05:32:40.263346 40506 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedvpc.example.com" } I0811 05:32:40.263435 40506 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0811 05:32:40.263463 40506 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedvpc.example.com" } I0811 05:32:40.263552 40506 iamrole.go:180] ListAttachedRolePolicies: masters.sharedvpc.example.com I0811 05:32:40.263580 40506 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0811 05:32:40.263636 40506 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedvpc.example.com" } iam-role:masters.sharedvpc.example.com ok I0811 05:32:40.263705 40506 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0811 05:32:40.263755 40506 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedvpc.example.com" } iam-role:nodes.sharedvpc.example.com ok I0811 05:32:40.263849 40506 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0811 05:32:40.264157 40506 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedVPC (0.36s) === RUN TestLifecycleComplex I0811 05:32:40.264529 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:40.264829 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:40.264889 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:40.264958 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:40.265032 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:40.265092 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:40.265149 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:40.265209 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:40.265255 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:40.265338 40506 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster complex.example.com I0811 05:32:40.300474 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0811 05:32:40.304671 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:40.304795 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:40.342023 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:40.537803 40506 iamrole.go:89] ListRoles: { } I0811 05:32:40.542694 40506 executor.go:111] Tasks: 0 done / 120 total; 51 can run I0811 05:32:40.543173 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } W0811 05:32:40.543855 40506 api.go:53] PageSize not implemented I0811 05:32:40.543387 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:40.544165 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } I0811 05:32:40.544252 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0811 05:32:40.544276 40506 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" } ] } I0811 05:32:40.544307 40506 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }] } I0811 05:32:40.544313 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:40.544413 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } I0811 05:32:40.544463 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:40.544571 40506 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com", Tags: [ { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] } I0811 05:32:40.544803 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0811 05:32:40.544978 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0811 05:32:40.545083 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:40.545271 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] }] } I0811 05:32:40.545490 40506 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-events.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/events", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0811 05:32:40.545668 40506 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }] } I0811 05:32:40.545846 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0811 05:32:40.545905 40506 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/main", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-main.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0811 05:32:40.546058 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0811 05:32:40.546107 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0811 05:32:40.546167 40506 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0811 05:32:40.546204 40506 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0811 05:32:40.546232 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0811 05:32:40.546276 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:40.547137 40506 executor.go:111] Tasks: 51 done / 120 total; 27 can run I0811 05:32:40.547268 40506 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0811 05:32:40.547346 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0811 05:32:40.547494 40506 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0811 05:32:40.547522 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0811 05:32:40.547465 40506 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-complex-example-com-vpjolq", Port: 443, Protocol: "TCP", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "tcp-complex-example-com-vpjolq" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0811 05:32:40.547259 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0811 05:32:40.547605 40506 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0811 05:32:40.547611 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:40.547684 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0811 05:32:40.547773 40506 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tls-complex-example-com-5nursn", Port: 443, Protocol: "TLS", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "tls-complex-example-com-5nursn" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0811 05:32:40.547851 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:40.547858 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.96.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-utility.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Utility" } ] }], VpcId: "vpc-1" } I0811 05:32:40.548024 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:32:40.548080 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:32:40.548052 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0811 05:32:40.548121 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0811 05:32:40.548130 40506 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0811 05:32:40.548222 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0811 05:32:40.548268 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:40.548338 40506 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0811 05:32:40.548444 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0811 05:32:40.548505 40506 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] } I0811 05:32:40.548608 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0811 05:32:40.548716 40506 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.1.0.0/16", VpcId: "vpc-1" } I0811 05:32:40.548766 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:40.548857 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0811 05:32:40.548934 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-private.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0811 05:32:40.549044 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0811 05:32:40.549122 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0811 05:32:40.549351 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I0811 05:32:40.549462 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0811 05:32:40.549523 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:40.549581 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0811 05:32:40.549632 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0811 05:32:40.549670 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:40.549751 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:40.549848 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:40.549927 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:40.549995 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:40.550094 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "api-elb.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I0811 05:32:40.550210 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:40.549854 40506 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0811 05:32:40.550252 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0811 05:32:40.550384 40506 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0811 05:32:40.550381 40506 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I0811 05:32:40.550456 40506 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.2.0.0/16", VpcId: "vpc-1" } I0811 05:32:40.550497 40506 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0811 05:32:40.550544 40506 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "complex.example.com" } ] }] } I0811 05:32:40.550645 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0811 05:32:40.550701 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0811 05:32:40.550758 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0811 05:32:40.550804 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0811 05:32:40.550862 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0811 05:32:40.550909 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0811 05:32:40.550956 40506 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" } ] } I0811 05:32:40.553015 40506 executor.go:111] Tasks: 78 done / 120 total; 39 can run I0811 05:32:40.553236 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:40.553360 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:40.553440 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0811 05:32:40.553531 40506 loadbalancers.go:34] PageSize not implemented I0811 05:32:40.553476 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:40.553642 40506 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "api-complex-example-com-vd3t5n", SubnetMappings: [{ AllocationId: "eipalloc-012345a678b9cdefa", SubnetId: "subnet-2" }], Tags: [ { Key: "Name", Value: "api.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ], Type: "network" } I0811 05:32:40.553730 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0811 05:32:40.553791 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0811 05:32:40.553771 40506 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-test-1:000000000000:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0811 05:32:40.553834 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:40.553862 40506 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0811 05:32:40.553927 40506 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [ { Key: "load_balancing.cross_zone.enabled", Value: "true" }, { Key: "access_logs.s3.enabled", Value: "true" }, { Key: "access_logs.s3.bucket", Value: "access-log-example" }, { Key: "access_logs.s3.prefix", Value: "" } ], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0811 05:32:40.553967 40506 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0811 05:32:40.554091 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0811 05:32:40.554213 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0811 05:32:40.554311 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-udp-1to65535-masters.complex.example.com" } ] }] } I0811 05:32:40.554691 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:40.555006 40506 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0811 05:32:40.555113 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:40.555248 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:40.555381 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0811 05:32:40.555425 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0811 05:32:40.555542 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-4003to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0811 05:32:40.555729 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0811 05:32:40.555896 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-::/0" } ] }] } I0811 05:32:40.556049 40506 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", RouteTableId: "rtb-2", TransitGatewayId: "tgw-123456" } I0811 05:32:40.556098 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }] } I0811 05:32:40.556321 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0811 05:32:40.556515 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:40.556618 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }] } I0811 05:32:40.556718 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }] } I0811 05:32:40.556855 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0811 05:32:40.556999 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:40.557079 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0811 05:32:40.557279 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0811 05:32:40.557325 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0811 05:32:40.557370 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0811 05:32:40.557640 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-3" } I0811 05:32:40.557713 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0811 05:32:40.558041 40506 routetable.go:149] CreateRoute: { DestinationCidrBlock: "192.168.1.10/32", RouteTableId: "rtb-2", TransitGatewayId: "tgw-0123456" } I0811 05:32:40.558142 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0811 05:32:40.558318 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:40.558453 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.0/24-ingress-tcp-443to443-masters.complex.example.com" } ] }] } I0811 05:32:40.558661 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0811 05:32:40.558818 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0811 05:32:40.558917 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-1to2379-masters.complex.example.com" } ] }] } I0811 05:32:40.559094 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }] } I0811 05:32:40.559197 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:40.559355 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0811 05:32:40.559560 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0811 05:32:40.559712 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }] } I0811 05:32:40.559817 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0811 05:32:40.559977 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-2382to4000-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0811 05:32:40.560130 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.complex.example.com" } ] }] } I0811 05:32:40.561197 40506 executor.go:111] Tasks: 117 done / 120 total; 3 can run I0811 05:32:40.561382 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:40.561571 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:40.561700 40506 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0811 05:32:40.561744 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:40.561778 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:40.561828 40506 dns.go:238] Pre-creating DNS records I0811 05:32:40.561853 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:40.571634 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0811 05:32:40.578721 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:40.578838 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:40.616113 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:40.796903 40506 iamrole.go:89] ListRoles: { } I0811 05:32:40.799523 40506 executor.go:111] Tasks: 0 done / 120 total; 51 can run I0811 05:32:40.799748 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:40.799854 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:40.799898 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0811 05:32:40.800109 40506 api.go:53] PageSize not implemented I0811 05:32:40.800195 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0811 05:32:40.800267 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0811 05:32:40.800329 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:40.800635 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0811 05:32:40.800769 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:40.800826 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0811 05:32:40.800929 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:40.801061 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0811 05:32:40.801164 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0811 05:32:40.802721 40506 executor.go:111] Tasks: 51 done / 120 total; 27 can run I0811 05:32:40.802864 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:40.803027 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0811 05:32:40.803215 40506 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0811 05:32:40.803160 40506 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0811 05:32:40.803279 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0811 05:32:40.803284 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:40.803291 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0811 05:32:40.803370 40506 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0811 05:32:40.803461 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0811 05:32:40.803371 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0811 05:32:40.803592 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0811 05:32:40.803657 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:40.803800 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] }] } I0811 05:32:40.803960 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:32:40.804011 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:40.804222 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0811 05:32:40.804270 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:40.804371 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:40.804458 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:40.804598 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:40.804685 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:40.804819 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:40.804964 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0811 05:32:40.805012 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0811 05:32:40.805223 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:40.808102 40506 executor.go:111] Tasks: 78 done / 120 total; 39 can run I0811 05:32:40.808247 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:40.808361 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:40.808455 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0811 05:32:40.808528 40506 loadbalancers.go:34] PageSize not implemented I0811 05:32:40.808546 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:40.808595 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:40.808633 40506 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0811 05:32:40.808674 40506 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0811 05:32:40.808711 40506 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0811 05:32:40.808744 40506 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0811 05:32:40.808837 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:40.809235 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:40.809377 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:40.809511 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:40.809623 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:40.809723 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:40.810068 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:40.810234 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:40.810382 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:40.813114 40506 executor.go:111] Tasks: 117 done / 120 total; 3 can run I0811 05:32:40.813230 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:40.813331 40506 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0811 05:32:40.813361 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0811 05:32:40.813399 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:40.813515 40506 executor.go:111] Tasks: 120 done / 120 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.sslCertificate=arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678] I0811 05:32:40.819820 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:40.823512 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:40.835317 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0811 05:32:40.839324 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:40.839466 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:40.876028 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:41.071374 40506 iamrole.go:89] ListRoles: { } I0811 05:32:41.073590 40506 executor.go:111] Tasks: 0 done / 120 total; 51 can run I0811 05:32:41.073793 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.073952 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0811 05:32:41.074070 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0811 05:32:41.074310 40506 api.go:53] PageSize not implemented I0811 05:32:41.074227 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:41.074265 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0811 05:32:41.074398 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0811 05:32:41.074511 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.074637 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0811 05:32:41.074690 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0811 05:32:41.074769 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:41.074831 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0811 05:32:41.074934 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.079519 40506 executor.go:111] Tasks: 51 done / 120 total; 27 can run I0811 05:32:41.079757 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0811 05:32:41.080458 40506 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0811 05:32:41.080537 40506 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0811 05:32:41.080662 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0811 05:32:41.079756 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:41.080773 40506 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0811 05:32:41.080893 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0811 05:32:41.080852 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.081112 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:32:41.081181 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I0811 05:32:41.081403 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.081621 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.081854 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0811 05:32:41.081932 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:41.082086 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.082241 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:41.082332 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0811 05:32:41.082518 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:41.082586 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:41.082677 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0811 05:32:41.082864 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:41.082941 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0811 05:32:41.083129 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0811 05:32:41.083176 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I0811 05:32:41.083389 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0811 05:32:41.086082 40506 executor.go:111] Tasks: 78 done / 120 total; 39 can run I0811 05:32:41.086236 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:41.086452 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.086744 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.086740 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0811 05:32:41.086955 40506 loadbalancers.go:34] PageSize not implemented I0811 05:32:41.086978 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.086985 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:41.087174 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:41.087245 40506 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0811 05:32:41.087290 40506 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0811 05:32:41.087327 40506 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0811 05:32:41.087360 40506 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0811 05:32:41.087449 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:41.087556 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { Names: ["api-complex-example-com-vd3t5n"] } I0811 05:32:41.087602 40506 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0811 05:32:41.087633 40506 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/2" } I0811 05:32:41.087607 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:41.087667 40506 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/1" } I0811 05:32:41.087700 40506 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0811 05:32:41.087799 40506 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0811 05:32:41.087861 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:41.087898 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:41.087861 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:41.088139 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:41.088235 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:41.088319 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:41.088510 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.091252 40506 executor.go:111] Tasks: 117 done / 120 total; 3 can run I0811 05:32:41.091441 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:41.091541 40506 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0811 05:32:41.091568 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0811 05:32:41.091594 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:41.091690 40506 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0811 05:32:41.091732 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:41.091765 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:41.091814 40506 dns.go:238] Pre-creating DNS records I0811 05:32:41.099594 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0811 05:32:41.103618 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.103753 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.140842 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:41.327609 40506 iamrole.go:89] ListRoles: { } I0811 05:32:41.329548 40506 executor.go:111] Tasks: 0 done / 120 total; 51 can run I0811 05:32:41.329724 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0811 05:32:41.329877 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0811 05:32:41.330025 40506 api.go:53] PageSize not implemented I0811 05:32:41.330118 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.330260 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:41.330303 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0811 05:32:41.330499 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0811 05:32:41.330681 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0811 05:32:41.330843 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.331041 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0811 05:32:41.331096 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0811 05:32:41.331168 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.331385 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:41.332689 40506 executor.go:111] Tasks: 51 done / 120 total; 27 can run I0811 05:32:41.332835 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:41.333003 40506 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0811 05:32:41.332983 40506 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0811 05:32:41.333086 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0811 05:32:41.332970 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.333176 40506 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0811 05:32:41.333237 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0811 05:32:41.333243 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0811 05:32:41.333201 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.333382 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:32:41.333440 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0811 05:32:41.333634 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:41.333704 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:41.333798 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0811 05:32:41.333946 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0811 05:32:41.333998 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0811 05:32:41.334138 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0811 05:32:41.334297 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:41.334340 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.334439 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.334539 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.334642 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:41.334722 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:41.334887 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0811 05:32:41.334964 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0811 05:32:41.338231 40506 executor.go:111] Tasks: 78 done / 120 total; 39 can run I0811 05:32:41.338390 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0811 05:32:41.338481 40506 loadbalancers.go:34] PageSize not implemented I0811 05:32:41.338502 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:41.338548 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:41.338595 40506 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0811 05:32:41.338665 40506 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0811 05:32:41.338726 40506 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0811 05:32:41.338571 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:41.338781 40506 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0811 05:32:41.338940 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.339185 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.339351 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:41.339767 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:41.339984 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.340101 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:41.340209 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:41.340283 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:41.340378 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:41.340486 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.347475 40506 executor.go:111] Tasks: 117 done / 120 total; 3 can run I0811 05:32:41.347652 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:41.347804 40506 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0811 05:32:41.347835 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0811 05:32:41.347863 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:41.347959 40506 executor.go:111] Tasks: 120 done / 120 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.additionalSecurityGroups=sg-123456] I0811 05:32:41.351728 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:41.356184 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.368927 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0811 05:32:41.372928 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.373030 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.409396 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:41.599146 40506 iamrole.go:89] ListRoles: { } I0811 05:32:41.601363 40506 executor.go:111] Tasks: 0 done / 121 total; 52 can run I0811 05:32:41.601545 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:41.601660 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } W0811 05:32:41.601838 40506 api.go:53] PageSize not implemented I0811 05:32:41.601870 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0811 05:32:41.601990 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.602242 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.602494 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0811 05:32:41.602699 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0811 05:32:41.602893 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.603134 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.603320 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0811 05:32:41.603368 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0811 05:32:41.603424 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0811 05:32:41.603603 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:41.604460 40506 executor.go:111] Tasks: 52 done / 121 total; 27 can run I0811 05:32:41.604616 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.604640 40506 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0811 05:32:41.604767 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0811 05:32:41.604704 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0811 05:32:41.604767 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0811 05:32:41.604882 40506 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0811 05:32:41.604888 40506 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0811 05:32:41.604924 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0811 05:32:41.604991 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0811 05:32:41.605059 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.605280 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:32:41.605336 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0811 05:32:41.605512 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:41.605582 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.605790 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0811 05:32:41.605840 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:41.605927 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0811 05:32:41.606053 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:41.606132 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.606231 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0811 05:32:41.606269 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:41.606345 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:41.606419 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0811 05:32:41.606516 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:41.606601 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0811 05:32:41.609961 40506 executor.go:111] Tasks: 79 done / 121 total; 39 can run I0811 05:32:41.610133 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.610308 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0811 05:32:41.610408 40506 loadbalancers.go:34] PageSize not implemented I0811 05:32:41.610323 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:41.610438 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:41.610506 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:41.610567 40506 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0811 05:32:41.610497 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.610628 40506 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0811 05:32:41.610670 40506 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0811 05:32:41.610709 40506 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0811 05:32:41.610732 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.610883 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:41.611109 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:41.611241 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:41.611371 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.611516 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:41.611805 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:41.611906 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:41.614398 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.614479 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.614666 40506 executor.go:111] Tasks: 118 done / 121 total; 3 can run I0811 05:32:41.614757 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:41.614840 40506 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0811 05:32:41.614867 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0811 05:32:41.614894 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:41.614979 40506 executor.go:111] Tasks: 121 done / 121 total; 0 can run I0811 05:32:41.615022 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:41.615054 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:41.615104 40506 dns.go:238] Pre-creating DNS records I0811 05:32:41.622958 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0811 05:32:41.626986 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.627091 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.664893 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:41.852339 40506 iamrole.go:89] ListRoles: { } I0811 05:32:41.854172 40506 executor.go:111] Tasks: 0 done / 121 total; 52 can run I0811 05:32:41.854493 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.854847 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } W0811 05:32:41.856016 40506 api.go:53] PageSize not implemented I0811 05:32:41.857030 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:41.857329 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0811 05:32:41.857520 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0811 05:32:41.857677 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0811 05:32:41.857804 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0811 05:32:41.857961 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0811 05:32:41.858016 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0811 05:32:41.858073 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0811 05:32:41.858456 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0811 05:32:41.858709 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.859472 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:41.862009 40506 executor.go:111] Tasks: 52 done / 121 total; 27 can run I0811 05:32:41.862172 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0811 05:32:41.862256 40506 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0811 05:32:41.862379 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0811 05:32:41.862378 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:41.862474 40506 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0811 05:32:41.862454 40506 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0811 05:32:41.862531 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0811 05:32:41.862514 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.862600 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0811 05:32:41.862642 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:41.862752 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:41.862850 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.863006 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:41.863085 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.863224 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.863374 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0811 05:32:41.863415 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:41.863499 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0811 05:32:41.863672 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:41.863762 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0811 05:32:41.863999 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0811 05:32:41.864061 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.864217 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:41.864375 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:32:41.864419 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0811 05:32:41.867658 40506 executor.go:111] Tasks: 79 done / 121 total; 39 can run I0811 05:32:41.867978 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:41.868147 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0811 05:32:41.868275 40506 loadbalancers.go:34] PageSize not implemented I0811 05:32:41.868293 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:41.868166 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:41.868353 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:41.868393 40506 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0811 05:32:41.868433 40506 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0811 05:32:41.868403 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:41.868467 40506 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0811 05:32:41.868501 40506 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0811 05:32:41.868591 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:41.868690 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:41.868875 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:41.869048 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.869154 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:41.869348 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.869420 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.869723 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.872863 40506 executor.go:111] Tasks: 118 done / 121 total; 3 can run I0811 05:32:41.873065 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:41.873167 40506 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0811 05:32:41.873194 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0811 05:32:41.873225 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:41.873341 40506 executor.go:111] Tasks: 121 done / 121 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t3.large] I0811 05:32:41.878580 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:41.882201 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.882296 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.891955 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0811 05:32:41.895984 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.896084 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.896146 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:41.932887 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:42.124123 40506 iamrole.go:89] ListRoles: { } I0811 05:32:42.126382 40506 executor.go:111] Tasks: 0 done / 121 total; 52 can run I0811 05:32:42.126641 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0811 05:32:42.126821 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0811 05:32:42.126971 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.127045 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:42.127197 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0811 05:32:42.127360 40506 api.go:53] PageSize not implemented I0811 05:32:42.127466 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0811 05:32:42.127604 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.127782 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0811 05:32:42.127888 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0811 05:32:42.127983 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.128091 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0811 05:32:42.128124 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0811 05:32:42.128183 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:42.129753 40506 executor.go:111] Tasks: 52 done / 121 total; 27 can run I0811 05:32:42.129908 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.130070 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.130238 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0811 05:32:42.130199 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0811 05:32:42.130290 40506 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0811 05:32:42.130170 40506 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0811 05:32:42.130390 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0811 05:32:42.130352 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:42.130468 40506 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0811 05:32:42.130503 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0811 05:32:42.130532 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0811 05:32:42.130577 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:42.130687 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.130836 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:42.130933 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0811 05:32:42.131089 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:42.131153 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.131292 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0811 05:32:42.131346 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:32:42.131432 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0811 05:32:42.131605 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0811 05:32:42.132843 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0811 05:32:42.132924 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:42.133065 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.133264 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:42.140636 40506 executor.go:111] Tasks: 79 done / 121 total; 39 can run I0811 05:32:42.140802 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:42.140981 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0811 05:32:42.141127 40506 loadbalancers.go:34] PageSize not implemented I0811 05:32:42.141130 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:42.141158 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:42.141275 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:42.141326 40506 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0811 05:32:42.141370 40506 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0811 05:32:42.141406 40506 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0811 05:32:42.141329 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.141440 40506 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0811 05:32:42.141528 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.141693 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.141899 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.142066 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:42.142436 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:42.142815 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:42.143008 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:42.143216 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:42.144956 40506 executor.go:111] Tasks: 118 done / 121 total; 3 can run I0811 05:32:42.145083 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:42.145220 40506 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0811 05:32:42.145287 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0811 05:32:42.145345 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:42.145510 40506 executor.go:111] Tasks: 121 done / 121 total; 0 can run I0811 05:32:42.145607 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:42.145705 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:42.145832 40506 dns.go:238] Pre-creating DNS records I0811 05:32:42.154050 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0811 05:32:42.158159 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.158290 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.158382 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.195458 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:42.392277 40506 iamrole.go:89] ListRoles: { } I0811 05:32:42.394549 40506 executor.go:111] Tasks: 0 done / 121 total; 52 can run I0811 05:32:42.394766 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:42.395012 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0811 05:32:42.395145 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } W0811 05:32:42.395438 40506 api.go:53] PageSize not implemented I0811 05:32:42.395328 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.395520 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0811 05:32:42.395706 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0811 05:32:42.395840 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.396017 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0811 05:32:42.396125 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.396262 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0811 05:32:42.396362 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0811 05:32:42.396400 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0811 05:32:42.396469 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:42.397940 40506 executor.go:111] Tasks: 52 done / 121 total; 27 can run I0811 05:32:42.398091 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:42.398201 40506 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0811 05:32:42.398256 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0811 05:32:42.398389 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0811 05:32:42.398438 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0811 05:32:42.398321 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0811 05:32:42.398488 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.398548 40506 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0811 05:32:42.398631 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0811 05:32:42.398568 40506 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0811 05:32:42.398668 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.398810 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:42.398912 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0811 05:32:42.399103 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:42.399167 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0811 05:32:42.399313 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:42.399391 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I0811 05:32:42.399536 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0811 05:32:42.399599 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:42.399688 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.399820 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0811 05:32:42.399872 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.400060 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:42.400171 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.400307 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:32:42.403608 40506 executor.go:111] Tasks: 79 done / 121 total; 39 can run I0811 05:32:42.403964 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:42.404087 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:42.404102 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0811 05:32:42.404273 40506 loadbalancers.go:34] PageSize not implemented I0811 05:32:42.404295 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:42.404350 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:42.404338 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:42.404387 40506 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0811 05:32:42.404430 40506 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0811 05:32:42.404464 40506 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0811 05:32:42.404495 40506 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0811 05:32:42.404486 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:42.404646 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:42.404758 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:42.404899 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.405082 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.405228 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:42.405322 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.405455 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.409048 40506 executor.go:111] Tasks: 118 done / 121 total; 3 can run I0811 05:32:42.409213 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:42.409364 40506 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0811 05:32:42.409431 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0811 05:32:42.409488 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:42.409680 40506 executor.go:111] Tasks: 121 done / 121 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t2.medium] I0811 05:32:42.415785 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:42.419732 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.419833 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.419983 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.430243 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0811 05:32:42.434160 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.434261 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.434324 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.434378 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.470777 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:42.663048 40506 iamrole.go:89] ListRoles: { } I0811 05:32:42.665405 40506 executor.go:111] Tasks: 0 done / 121 total; 52 can run W0811 05:32:42.665973 40506 api.go:53] PageSize not implemented I0811 05:32:42.665887 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.666113 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0811 05:32:42.666224 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0811 05:32:42.666312 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:42.666363 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0811 05:32:42.666471 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.666627 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.666738 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0811 05:32:42.666771 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0811 05:32:42.666826 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:42.666876 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.667068 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0811 05:32:42.667200 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0811 05:32:42.668668 40506 executor.go:111] Tasks: 52 done / 121 total; 27 can run I0811 05:32:42.668844 40506 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0811 05:32:42.668959 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0811 05:32:42.668914 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.669098 40506 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0811 05:32:42.669117 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0811 05:32:42.669174 40506 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0811 05:32:42.669271 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0811 05:32:42.669304 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0811 05:32:42.669260 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.669474 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0811 05:32:42.669528 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.669674 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:42.669771 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:42.669857 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:42.669939 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:32:42.669985 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0811 05:32:42.670141 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0811 05:32:42.670192 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:42.670335 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0811 05:32:42.670528 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:42.670619 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I0811 05:32:42.670813 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.670945 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.671086 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0811 05:32:42.671129 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:42.674876 40506 executor.go:111] Tasks: 79 done / 121 total; 39 can run I0811 05:32:42.675130 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:42.675254 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:42.675396 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:42.675400 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0811 05:32:42.675488 40506 loadbalancers.go:34] PageSize not implemented I0811 05:32:42.675507 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:42.675557 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:42.675601 40506 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0811 05:32:42.675638 40506 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0811 05:32:42.675674 40506 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0811 05:32:42.675623 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.675728 40506 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0811 05:32:42.675846 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.676053 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.676231 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.676305 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:42.676419 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:42.676513 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:42.676663 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:42.678512 40506 executor.go:111] Tasks: 118 done / 121 total; 3 can run I0811 05:32:42.678604 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:42.678690 40506 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0811 05:32:42.678718 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0811 05:32:42.678745 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:42.678867 40506 executor.go:111] Tasks: 121 done / 121 total; 0 can run I0811 05:32:42.678921 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:42.678959 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:42.679041 40506 dns.go:238] Pre-creating DNS records I0811 05:32:42.687047 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0811 05:32:42.691133 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.691238 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.691301 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.691361 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.728542 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:42.919782 40506 iamrole.go:89] ListRoles: { } I0811 05:32:42.922416 40506 executor.go:111] Tasks: 0 done / 121 total; 52 can run I0811 05:32:42.922626 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } W0811 05:32:42.923054 40506 api.go:53] PageSize not implemented I0811 05:32:42.922977 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0811 05:32:42.923092 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:42.923213 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0811 05:32:42.923359 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.923516 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.923667 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0811 05:32:42.923770 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0811 05:32:42.923865 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.923982 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0811 05:32:42.924018 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0811 05:32:42.924055 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.924274 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:42.925890 40506 executor.go:111] Tasks: 52 done / 121 total; 27 can run I0811 05:32:42.926216 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0811 05:32:42.926113 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.926292 40506 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0811 05:32:42.926257 40506 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0811 05:32:42.926354 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0811 05:32:42.926312 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.926461 40506 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0811 05:32:42.926557 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0811 05:32:42.926575 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:32:42.926653 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0811 05:32:42.926879 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0811 05:32:42.927144 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0811 05:32:42.927226 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:42.927372 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I0811 05:32:42.927531 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0811 05:32:42.927708 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:42.927797 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.928004 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0811 05:32:42.928058 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.928322 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0811 05:32:42.928391 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:42.928533 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:42.928667 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0811 05:32:42.928867 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:42.929031 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:42.931625 40506 executor.go:111] Tasks: 79 done / 121 total; 39 can run I0811 05:32:42.931847 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:42.932043 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0811 05:32:42.932135 40506 loadbalancers.go:34] PageSize not implemented I0811 05:32:42.932108 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:42.932154 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:42.932211 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:42.932255 40506 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0811 05:32:42.932296 40506 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0811 05:32:42.932331 40506 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0811 05:32:42.932392 40506 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0811 05:32:42.932270 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.932565 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.932936 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.933147 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:42.933281 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:42.933389 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:42.933503 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:42.933665 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:42.933859 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.935195 40506 executor.go:111] Tasks: 118 done / 121 total; 3 can run I0811 05:32:42.935377 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:42.935474 40506 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0811 05:32:42.935509 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0811 05:32:42.935539 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:42.935633 40506 executor.go:111] Tasks: 121 done / 121 total; 0 can run I0811 05:32:42.935969 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0811 05:32:42.936027 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0811 05:32:42.936067 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0811 05:32:42.936103 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0811 05:32:42.936142 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0811 05:32:42.936177 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0811 05:32:42.936210 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0811 05:32:42.936247 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0811 05:32:42.936289 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0811 05:32:42.936331 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0811 05:32:42.936377 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0811 05:32:42.936413 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0811 05:32:42.936448 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0811 05:32:42.936489 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0811 05:32:42.936524 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0811 05:32:42.936567 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0811 05:32:42.936606 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0811 05:32:42.936645 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0811 05:32:42.936683 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:32:42.936724 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0811 05:32:42.936760 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0811 05:32:42.936800 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0811 05:32:42.936840 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0811 05:32:42.936874 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0811 05:32:42.936909 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0811 05:32:42.936951 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0811 05:32:42.936990 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0811 05:32:42.937029 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0811 05:32:42.939722 40506 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0811 05:32:42.939758 40506 keypairs.go:95] DescribeKeyPairs: { } I0811 05:32:42.939794 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.939855 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0811 05:32:42.939922 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.939984 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.940034 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.940094 40506 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.940137 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.940184 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0811 05:32:42.940243 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.940318 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0811 05:32:42.940363 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } W0811 05:32:42.940419 40506 api.go:53] PageSize not implemented I0811 05:32:42.940440 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0811 05:32:42.940467 40506 loadbalancers.go:34] PageSize not implemented I0811 05:32:42.940482 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0811 05:32:42.940522 40506 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0811 05:32:42.940547 40506 targetgroups.go:35] PageSize not implemented I0811 05:32:42.940565 40506 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5" ] } I0811 05:32:42.940640 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:42.940668 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:42.940729 40506 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0811 05:32:42.940767 40506 iamrole.go:89] ListRoles: { } I0811 05:32:42.940811 40506 internetgateways.go:101] DescribeInternetGateways: { } I0811 05:32:42.940863 40506 routetable.go:53] DescribeRouteTables: { } I0811 05:32:42.940913 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0811 05:32:42.941480 40506 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } autoscaling-group:master-us-test-1a.masters.complex.example.com ok autoscaling-group:nodes.complex.example.com ok I0811 05:32:42.941550 40506 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-1 ok I0811 05:32:42.941631 40506 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I0811 05:32:42.941604 40506 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4" } I0811 05:32:42.941685 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0811 05:32:42.941713 40506 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0811 05:32:42.941702 40506 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0811 05:32:42.941773 40506 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4 ok I0811 05:32:42.941774 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1 ok I0811 05:32:42.941615 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } iam-instance-profile:nodes.complex.example.com ok I0811 05:32:42.941819 40506 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } route53-record:Z1AFAKE1ZON3YO/A/api.complex.example.com ok volume:vol-2 ok I0811 05:32:42.941808 40506 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0811 05:32:42.941866 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:42.941900 40506 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } iam-instance-profile:masters.complex.example.com ok I0811 05:32:42.941812 40506 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5 ok I0811 05:32:42.941925 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 } ] } W0811 05:32:42.942144 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:42.942168 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-2 ok I0811 05:32:42.942231 40506 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0811 05:32:42.942281 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:42.942327 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }, { FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0811 05:32:42.942688 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:42.942710 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0811 05:32:42.942754 40506 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0811 05:32:42.942855 40506 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.complex.example.com" } I0811 05:32:42.942910 40506 iamrole.go:180] ListAttachedRolePolicies: nodes.complex.example.com I0811 05:32:42.942886 40506 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0811 05:32:42.942928 40506 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0811 05:32:42.942989 40506 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0811 05:32:42.943013 40506 iamrole.go:137] DeleteRole: { RoleName: "nodes.complex.example.com" } iam-role:nodes.complex.example.com ok I0811 05:32:42.943074 40506 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.complex.example.com" } I0811 05:32:42.943136 40506 iamrole.go:180] ListAttachedRolePolicies: masters.complex.example.com I0811 05:32:42.943152 40506 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0811 05:32:42.943183 40506 iamrole.go:137] DeleteRole: { RoleName: "masters.complex.example.com" } iam-role:masters.complex.example.com ok I0811 05:32:42.943267 40506 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0811 05:32:42.943355 40506 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0811 05:32:42.943446 40506 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0811 05:32:42.943745 40506 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleComplex (2.68s) === RUN TestLifecycleExternalLB I0811 05:32:42.944113 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:42.944409 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:42.944459 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:42.944504 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:42.944555 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:42.944598 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:42.944632 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:42.944671 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:42.944700 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:42.944757 40506 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster externallb.example.com I0811 05:32:42.969622 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:42.972521 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:42.972627 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:43.009072 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:43.110691 40506 iamrole.go:89] ListRoles: { } I0811 05:32:43.112395 40506 executor.go:111] Tasks: 0 done / 92 total; 50 can run W0811 05:32:43.112647 40506 api.go:53] PageSize not implemented I0811 05:32:43.112690 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0811 05:32:43.112828 40506 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0811 05:32:43.113002 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0811 05:32:43.112964 40506 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "externallb.example.com" }] }] } I0811 05:32:43.113104 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0811 05:32:43.113111 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:43.113115 40506 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0811 05:32:43.113165 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } I0811 05:32:43.113265 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } W0811 05:32:43.113221 40506 api.go:53] PageSize not implemented I0811 05:32:43.113283 40506 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I0811 05:32:43.113240 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0811 05:32:43.113427 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:43.113376 40506 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0811 05:32:43.113527 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } W0811 05:32:43.113458 40506 api.go:53] PageSize not implemented I0811 05:32:43.113519 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:43.113627 40506 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0811 05:32:43.113615 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } I0811 05:32:43.113703 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0811 05:32:43.113742 40506 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I0811 05:32:43.113697 40506 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0811 05:32:43.113880 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0811 05:32:43.113942 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0811 05:32:43.114129 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0811 05:32:43.114320 40506 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0811 05:32:43.114387 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0811 05:32:43.114624 40506 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0811 05:32:43.115888 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0811 05:32:43.115998 40506 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0811 05:32:43.116047 40506 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0811 05:32:43.116083 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0811 05:32:43.116162 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:43.117229 40506 executor.go:111] Tasks: 50 done / 92 total; 19 can run I0811 05:32:43.117487 40506 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I0811 05:32:43.117423 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:43.117770 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0811 05:32:43.117665 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0811 05:32:43.118001 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I0811 05:32:43.118459 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0811 05:32:43.118214 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0811 05:32:43.119151 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0811 05:32:43.119260 40506 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0811 05:32:43.119322 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:43.119462 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["externallb.example.com"] }] } I0811 05:32:43.120139 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0811 05:32:43.120263 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0811 05:32:43.121333 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0811 05:32:43.121447 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:43.121405 40506 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0811 05:32:43.121491 40506 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0811 05:32:43.121556 40506 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0811 05:32:43.121643 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0811 05:32:43.121699 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0811 05:32:43.121638 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" } ] }], VpcId: "vpc-1" } I0811 05:32:43.121799 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:32:43.121831 40506 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] }], VpcId: "vpc-1" } I0811 05:32:43.121868 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0811 05:32:43.122084 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0811 05:32:43.122197 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:43.122292 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:32:43.122398 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0811 05:32:43.122483 40506 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] } I0811 05:32:43.123282 40506 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0811 05:32:43.127051 40506 executor.go:111] Tasks: 69 done / 92 total; 21 can run I0811 05:32:43.127211 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:43.127426 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:43.127579 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:43.127651 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-2382to4000-masters.externallb.example.com" }] }] } I0811 05:32:43.127986 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0811 05:32:43.128269 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0811 05:32:43.128457 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0811 05:32:43.128692 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:43.128819 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0811 05:32:43.128956 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-1to2379-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0811 05:32:43.129216 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0811 05:32:43.129368 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-4003to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0811 05:32:43.129539 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-udp-1to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0811 05:32:43.129694 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:43.129800 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0811 05:32:43.130029 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:43.130147 40506 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0811 05:32:43.130194 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0811 05:32:43.130248 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0811 05:32:43.130404 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0811 05:32:43.130569 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0811 05:32:43.130730 40506 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0811 05:32:43.130799 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.externallb.example.com" }] }] } I0811 05:32:43.130960 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-::/0" }] }] } I0811 05:32:43.131130 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:43.133669 40506 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0811 05:32:43.133993 40506 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0811 05:32:43.134050 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:43.134100 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:43.134163 40506 dns.go:238] Pre-creating DNS records I0811 05:32:43.134194 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:43.140358 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:43.143259 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:43.143365 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:43.180059 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:43.274037 40506 iamrole.go:89] ListRoles: { } I0811 05:32:43.275716 40506 executor.go:111] Tasks: 0 done / 92 total; 50 can run I0811 05:32:43.275924 40506 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0811 05:32:43.275914 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0811 05:32:43.276022 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0811 05:32:43.276055 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0811 05:32:43.276090 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0811 05:32:43.276158 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } W0811 05:32:43.276252 40506 api.go:53] PageSize not implemented I0811 05:32:43.276213 40506 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0811 05:32:43.276354 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0811 05:32:43.276309 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } W0811 05:32:43.276506 40506 api.go:53] PageSize not implemented I0811 05:32:43.276550 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:43.276464 40506 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } W0811 05:32:43.276617 40506 api.go:53] PageSize not implemented I0811 05:32:43.276641 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0811 05:32:43.276591 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:43.276650 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:43.276789 40506 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0811 05:32:43.276819 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0811 05:32:43.276902 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0811 05:32:43.276981 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] }] } I0811 05:32:43.279281 40506 executor.go:111] Tasks: 50 done / 92 total; 19 can run I0811 05:32:43.279448 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0811 05:32:43.279641 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:43.279703 40506 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I0811 05:32:43.279749 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I0811 05:32:43.279735 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0811 05:32:43.279869 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:43.279924 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:43.279996 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0811 05:32:43.280112 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:43.280162 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] }] } I0811 05:32:43.280273 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:32:43.280306 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0811 05:32:43.285752 40506 executor.go:111] Tasks: 69 done / 92 total; 21 can run I0811 05:32:43.285938 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:43.286137 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:43.286268 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:43.286417 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:43.286701 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:43.286846 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:43.287060 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:43.289251 40506 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0811 05:32:43.289488 40506 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0811 05:32:43.292730 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0811 05:32:43.292793 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0811 05:32:43.292830 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0811 05:32:43.292863 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0811 05:32:43.292906 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0811 05:32:43.292941 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0811 05:32:43.292976 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0811 05:32:43.293012 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0811 05:32:43.293049 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0811 05:32:43.293084 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0811 05:32:43.293122 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0811 05:32:43.293165 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0811 05:32:43.293199 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0811 05:32:43.293233 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0811 05:32:43.293269 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0811 05:32:43.293306 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0811 05:32:43.293342 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0811 05:32:43.293379 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:32:43.293416 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0811 05:32:43.293448 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0811 05:32:43.293481 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0811 05:32:43.293513 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0811 05:32:43.293551 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0811 05:32:43.293587 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0811 05:32:43.293624 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0811 05:32:43.294994 40506 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0811 05:32:43.295023 40506 keypairs.go:95] DescribeKeyPairs: { } I0811 05:32:43.295065 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0811 05:32:43.295114 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0811 05:32:43.295170 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0811 05:32:43.295231 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0811 05:32:43.295280 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0811 05:32:43.295334 40506 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0811 05:32:43.295376 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0811 05:32:43.295415 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0811 05:32:43.295463 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0811 05:32:43.295513 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0811 05:32:43.295552 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } W0811 05:32:43.295607 40506 api.go:53] PageSize not implemented I0811 05:32:43.295630 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0811 05:32:43.295658 40506 loadbalancers.go:34] PageSize not implemented I0811 05:32:43.295698 40506 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0811 05:32:43.295725 40506 targetgroups.go:35] PageSize not implemented I0811 05:32:43.295742 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0811 05:32:43.295796 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:43.295826 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:43.295877 40506 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0811 05:32:43.295915 40506 iamrole.go:89] ListRoles: { } I0811 05:32:43.295959 40506 internetgateways.go:101] DescribeInternetGateways: { } I0811 05:32:43.296003 40506 routetable.go:53] DescribeRouteTables: { } I0811 05:32:43.296036 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-config:lt-2 ok autoscaling-group:master-us-test-1a.masters.externallb.example.com ok autoscaling-group:nodes.externallb.example.com ok I0811 05:32:43.296892 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:43.296949 40506 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0811 05:32:43.297023 40506 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } iam-instance-profile:masters.externallb.example.com ok I0811 05:32:43.296955 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0811 05:32:43.297081 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:43.297060 40506 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0811 05:32:43.297100 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } I0811 05:32:43.297105 40506 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } security-group:sg-1 ok iam-instance-profile:nodes.externallb.example.com ok I0811 05:32:43.297140 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:43.297038 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.externallb.example.com. ok I0811 05:32:43.297203 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } W0811 05:32:43.297399 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:43.297419 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-1 ok I0811 05:32:43.297476 40506 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0811 05:32:43.297538 40506 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0811 05:32:43.297575 40506 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0811 05:32:43.297644 40506 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0811 05:32:43.297712 40506 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0811 05:32:43.297799 40506 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.externallb.example.com" } I0811 05:32:43.297846 40506 iamrole.go:180] ListAttachedRolePolicies: nodes.externallb.example.com I0811 05:32:43.297861 40506 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0811 05:32:43.297824 40506 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } I0811 05:32:43.297897 40506 iamrole.go:137] DeleteRole: { RoleName: "nodes.externallb.example.com" } subnet:subnet-1 ok iam-role:nodes.externallb.example.com ok I0811 05:32:43.297929 40506 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.externallb.example.com" } I0811 05:32:43.297991 40506 iamrole.go:180] ListAttachedRolePolicies: masters.externallb.example.com I0811 05:32:43.298004 40506 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0811 05:32:43.298031 40506 iamrole.go:137] DeleteRole: { RoleName: "masters.externallb.example.com" } iam-role:masters.externallb.example.com ok I0811 05:32:43.298098 40506 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0811 05:32:43.298176 40506 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0811 05:32:43.298251 40506 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0811 05:32:43.298505 40506 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleExternalLB (0.35s) === RUN TestLifecycleSharedSubnet I0811 05:32:43.298769 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:43.299071 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:43.299117 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:43.299159 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:43.299210 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:43.299251 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:43.299281 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:43.299319 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:43.299349 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:43.299403 40506 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedsubnet.example.com I0811 05:32:43.321183 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:43.323462 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:43.323535 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:43.360493 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:43.463274 40506 iamrole.go:89] ListRoles: { } I0811 05:32:43.464761 40506 executor.go:111] Tasks: 0 done / 78 total; 42 can run I0811 05:32:43.465193 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:43.465279 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0811 05:32:43.465770 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } I0811 05:32:43.465594 40506 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-events.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0811 05:32:43.465909 40506 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0811 05:32:43.465985 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0811 05:32:43.466026 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } I0811 05:32:43.466060 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:43.466106 40506 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "masters.sharedsubnet.example.com" }] } I0811 05:32:43.466169 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:43.466382 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0811 05:32:43.466472 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0811 05:32:43.466502 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0811 05:32:43.466553 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0811 05:32:43.466780 40506 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0811 05:32:43.466942 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0811 05:32:43.467013 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0811 05:32:43.470263 40506 executor.go:111] Tasks: 42 done / 78 total; 16 can run I0811 05:32:43.470733 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0811 05:32:43.470947 40506 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0811 05:32:43.470735 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0811 05:32:43.471011 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0811 05:32:43.471143 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0811 05:32:43.471241 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0811 05:32:43.471330 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "masters.sharedsubnet.example.com" }] }], VpcId: "vpc-12345678" } I0811 05:32:43.473376 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0811 05:32:43.473461 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0811 05:32:43.473576 40506 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] } I0811 05:32:43.473719 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:43.473723 40506 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0811 05:32:43.473794 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0811 05:32:43.473972 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0811 05:32:43.474270 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0811 05:32:43.474418 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0811 05:32:43.474491 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:43.475470 40506 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0811 05:32:43.479293 40506 executor.go:111] Tasks: 58 done / 78 total; 18 can run I0811 05:32:43.479507 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-1to2379-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0811 05:32:43.479714 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0811 05:32:43.479856 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-udp-1to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0811 05:32:43.480095 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0811 05:32:43.480263 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0811 05:32:43.480420 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0811 05:32:43.480711 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:43.480895 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:43.480964 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0811 05:32:43.481320 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0811 05:32:43.481761 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0811 05:32:43.481957 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-2382to4000-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0811 05:32:43.482212 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedsubnet.example.com" }] }] } I0811 05:32:43.482399 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-masters.sharedsubnet.example.com" }] }] } I0811 05:32:43.482538 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-4003to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0811 05:32:43.482678 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0811 05:32:43.482824 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:43.482936 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:43.484014 40506 executor.go:111] Tasks: 76 done / 78 total; 2 can run I0811 05:32:43.484290 40506 executor.go:111] Tasks: 78 done / 78 total; 0 can run I0811 05:32:43.484346 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:43.484397 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:43.484458 40506 dns.go:238] Pre-creating DNS records I0811 05:32:43.484488 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:43.490285 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:43.493223 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:43.493319 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:43.530965 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:43.634079 40506 iamrole.go:89] ListRoles: { } I0811 05:32:43.635589 40506 executor.go:111] Tasks: 0 done / 78 total; 42 can run I0811 05:32:43.636023 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0811 05:32:43.636287 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:43.636304 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0811 05:32:43.636498 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0811 05:32:43.636562 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0811 05:32:43.636611 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:43.636735 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:43.636932 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0811 05:32:43.638888 40506 executor.go:111] Tasks: 42 done / 78 total; 16 can run I0811 05:32:43.639125 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0811 05:32:43.639100 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0811 05:32:43.639237 40506 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0811 05:32:43.639248 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0811 05:32:43.639488 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:43.639578 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0811 05:32:43.639747 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0811 05:32:43.639915 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:43.645164 40506 executor.go:111] Tasks: 58 done / 78 total; 18 can run I0811 05:32:43.645387 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:43.645569 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:43.645636 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:43.645908 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:43.648200 40506 executor.go:111] Tasks: 76 done / 78 total; 2 can run I0811 05:32:43.648414 40506 executor.go:111] Tasks: 78 done / 78 total; 0 can run I0811 05:32:43.651236 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0811 05:32:43.651305 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0811 05:32:43.651343 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0811 05:32:43.651375 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0811 05:32:43.651412 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0811 05:32:43.651447 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0811 05:32:43.651479 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0811 05:32:43.651517 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0811 05:32:43.651562 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0811 05:32:43.651602 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0811 05:32:43.651634 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0811 05:32:43.651664 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0811 05:32:43.651697 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0811 05:32:43.651739 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0811 05:32:43.651774 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0811 05:32:43.651812 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0811 05:32:43.651847 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0811 05:32:43.651881 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0811 05:32:43.651923 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0811 05:32:43.651962 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0811 05:32:43.653409 40506 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0811 05:32:43.653443 40506 keypairs.go:95] DescribeKeyPairs: { } I0811 05:32:43.653485 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0811 05:32:43.653539 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0811 05:32:43.653597 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0811 05:32:43.653661 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0811 05:32:43.653708 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0811 05:32:43.653753 40506 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0811 05:32:43.653797 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0811 05:32:43.653835 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0811 05:32:43.653877 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0811 05:32:43.653924 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0811 05:32:43.653960 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } W0811 05:32:43.654011 40506 api.go:53] PageSize not implemented I0811 05:32:43.654034 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0811 05:32:43.654064 40506 loadbalancers.go:34] PageSize not implemented I0811 05:32:43.654086 40506 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0811 05:32:43.654111 40506 targetgroups.go:35] PageSize not implemented I0811 05:32:43.654130 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0811 05:32:43.654185 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:43.654215 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:43.654268 40506 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0811 05:32:43.654305 40506 iamrole.go:89] ListRoles: { } I0811 05:32:43.654353 40506 internetgateways.go:101] DescribeInternetGateways: { } I0811 05:32:43.654395 40506 routetable.go:53] DescribeRouteTables: { } autoscaling-group:master-us-test-1a.masters.sharedsubnet.example.com ok I0811 05:32:43.654805 40506 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0811 05:32:43.654869 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-group:nodes.sharedsubnet.example.com ok I0811 05:32:43.654851 40506 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0811 05:32:43.654963 40506 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } iam-instance-profile:masters.sharedsubnet.example.com ok I0811 05:32:43.654926 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0811 05:32:43.655056 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:43.655032 40506 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0811 05:32:43.655114 40506 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } iam-instance-profile:nodes.sharedsubnet.example.com ok I0811 05:32:43.655075 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } I0811 05:32:43.654876 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } security-group:sg-2 ok route53-record:Z1AFAKE1ZON3YO/A/api.internal.sharedsubnet.example.com. ok autoscaling-config:lt-2 ok autoscaling-config:lt-1 ok I0811 05:32:43.655285 40506 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0811 05:32:43.655362 40506 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0811 05:32:43.655418 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:43.655474 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0811 05:32:43.655703 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:43.655718 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0811 05:32:43.655810 40506 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedsubnet.example.com" } I0811 05:32:43.655887 40506 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0811 05:32:43.655909 40506 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0811 05:32:43.655945 40506 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedsubnet.example.com" } iam-role:nodes.sharedsubnet.example.com ok I0811 05:32:43.656003 40506 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedsubnet.example.com" } I0811 05:32:43.656058 40506 iamrole.go:180] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0811 05:32:43.656072 40506 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0811 05:32:43.656101 40506 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedsubnet.example.com" } iam-role:masters.sharedsubnet.example.com ok I0811 05:32:43.656360 40506 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedSubnet (0.36s) === RUN TestLifecyclePrivateSharedSubnet I0811 05:32:43.656667 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:43.656922 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:43.656972 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:43.657015 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:43.657063 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:43.657109 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:43.657142 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:43.657181 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:43.657208 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:43.657262 40506 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-subnet.example.com I0811 05:32:43.689792 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:43.693440 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:43.693542 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:43.693609 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:43.730184 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:43.827966 40506 iamrole.go:89] ListRoles: { } I0811 05:32:43.829789 40506 executor.go:111] Tasks: 0 done / 104 total; 45 can run I0811 05:32:43.831921 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0811 05:32:43.832152 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:43.832378 40506 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0811 05:32:43.832747 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0811 05:32:43.832859 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0811 05:32:43.833197 40506 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0811 05:32:43.833442 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0811 05:32:43.834008 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0811 05:32:43.834152 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0811 05:32:43.834204 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0811 05:32:43.834295 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:43.834301 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } I0811 05:32:43.834451 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } I0811 05:32:43.834617 40506 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0811 05:32:43.834735 40506 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0811 05:32:43.836675 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } I0811 05:32:43.836793 40506 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] } I0811 05:32:43.834452 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:43.837305 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0811 05:32:43.838602 40506 executor.go:111] Tasks: 45 done / 104 total; 24 can run I0811 05:32:43.838753 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0811 05:32:43.838773 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0811 05:32:43.838982 40506 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0811 05:32:43.838888 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0811 05:32:43.839086 40506 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0811 05:32:43.839170 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0811 05:32:43.839311 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0811 05:32:43.839400 40506 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0811 05:32:43.839535 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0811 05:32:43.839508 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I0811 05:32:43.839772 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0811 05:32:43.839881 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0811 05:32:43.839819 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0811 05:32:43.839989 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0811 05:32:43.839936 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:43.840050 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0811 05:32:43.840048 40506 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0811 05:32:43.840244 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0811 05:32:43.840372 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0811 05:32:43.840506 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0811 05:32:43.840550 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0811 05:32:43.840652 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0811 05:32:43.840701 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0811 05:32:43.840790 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0811 05:32:43.840838 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0811 05:32:43.840882 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0811 05:32:43.841069 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0811 05:32:43.841171 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0811 05:32:43.841251 40506 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" },{ Key: "SubnetType", Value: "Private" },{ Key: "kubernetes.io/role/internal-elb", Value: "1" }] } I0811 05:32:43.841374 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0811 05:32:43.841536 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0811 05:32:43.841609 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0811 05:32:43.841698 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:43.841750 40506 tags.go:42] CreateTags { Resources: ["subnet-abcdef"], Tags: [{ Key: "SubnetType", Value: "Utility" },{ Key: "kubernetes.io/role/elb", Value: "1" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" }] } I0811 05:32:43.841871 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0811 05:32:43.841809 40506 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0811 05:32:43.844729 40506 executor.go:111] Tasks: 69 done / 104 total; 31 can run I0811 05:32:43.844967 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } W0811 05:32:43.845089 40506 api.go:53] PageSize not implemented W0811 05:32:43.845147 40506 api.go:53] PageSize not implemented I0811 05:32:43.845123 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:43.845256 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0811 05:32:43.845374 40506 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"], Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "api.private-shared-subnet.example.com" }] } I0811 05:32:43.845468 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0811 05:32:43.845505 40506 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0811 05:32:43.845565 40506 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0811 05:32:43.845474 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0811 05:32:43.845700 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0811 05:32:43.845785 40506 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] } I0811 05:32:43.845716 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0811 05:32:43.845848 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0811 05:32:43.845882 40506 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0811 05:32:43.845934 40506 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0811 05:32:43.845877 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-2382to4000-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0811 05:32:43.846063 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-ingress-tcp-22to22-bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0811 05:32:43.846323 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:43.846436 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0811 05:32:43.846586 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0811 05:32:43.846739 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-nodes.private-shared-subnet.example.com" }] }] } I0811 05:32:43.846893 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0811 05:32:43.847040 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0811 05:32:43.847188 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } I0811 05:32:43.847329 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0811 05:32:43.847488 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:43.847633 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0811 05:32:43.847860 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-udp-1to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0811 05:32:43.848034 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-1to2379-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0811 05:32:43.848199 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-subnet.example.com" }] }] } I0811 05:32:43.848350 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0811 05:32:43.848481 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0811 05:32:43.848613 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0811 05:32:43.848751 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-4003to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0811 05:32:43.848886 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0811 05:32:43.849027 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:43.849152 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:43.849215 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0811 05:32:43.849424 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-subnet.example.com" }] }] } I0811 05:32:43.849571 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I0811 05:32:43.849693 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0811 05:32:43.849957 40506 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0811 05:32:43.850094 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:43.850245 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:43.850371 40506 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0811 05:32:43.850413 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:43.850456 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:43.850507 40506 dns.go:238] Pre-creating DNS records I0811 05:32:43.850532 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:43.856872 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:43.859893 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:43.859987 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:43.860050 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:43.896523 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:43.993210 40506 iamrole.go:89] ListRoles: { } I0811 05:32:43.995265 40506 executor.go:111] Tasks: 0 done / 104 total; 45 can run I0811 05:32:43.995923 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0811 05:32:43.996140 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:43.996154 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0811 05:32:43.996391 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:43.996472 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:43.996594 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0811 05:32:43.996670 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0811 05:32:43.996701 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0811 05:32:43.997845 40506 executor.go:111] Tasks: 45 done / 104 total; 24 can run I0811 05:32:43.997976 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0811 05:32:43.998037 40506 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0811 05:32:43.998098 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0811 05:32:43.998256 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0811 05:32:43.998365 40506 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0811 05:32:43.998288 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0811 05:32:43.998503 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0811 05:32:43.998585 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I0811 05:32:43.998792 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:43.998852 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:43.998910 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0811 05:32:43.999100 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0811 05:32:43.999155 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0811 05:32:43.999270 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0811 05:32:43.999378 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0811 05:32:43.999511 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0811 05:32:44.003013 40506 executor.go:111] Tasks: 69 done / 104 total; 31 can run I0811 05:32:44.003327 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } W0811 05:32:44.003409 40506 api.go:53] PageSize not implemented I0811 05:32:44.003517 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:44.003487 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I0811 05:32:44.003610 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } W0811 05:32:44.003651 40506 api.go:53] PageSize not implemented I0811 05:32:44.003689 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0811 05:32:44.003806 40506 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0811 05:32:44.003766 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:44.003894 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:44.003912 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0811 05:32:44.004007 40506 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0811 05:32:44.004044 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:44.004183 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:44.005657 40506 executor.go:111] Tasks: 100 done / 104 total; 4 can run W0811 05:32:44.005808 40506 api.go:53] PageSize not implemented I0811 05:32:44.005841 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I0811 05:32:44.005867 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:44.005952 40506 dnsname.go:121] AliasTarget for "api.private-shared-subnet.example.com" is "api-private-shared-subnet-n2f8ak.elb.cloudmock.com" I0811 05:32:44.005984 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } W0811 05:32:44.006035 40506 api.go:53] PageSize not implemented I0811 05:32:44.006065 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I0811 05:32:44.006254 40506 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0811 05:32:44.009901 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0811 05:32:44.009968 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0811 05:32:44.010006 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0811 05:32:44.010039 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0811 05:32:44.010077 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0811 05:32:44.010113 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0811 05:32:44.010147 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0811 05:32:44.010188 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0811 05:32:44.010227 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0811 05:32:44.010267 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0811 05:32:44.010313 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0811 05:32:44.010349 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0811 05:32:44.010381 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0811 05:32:44.010418 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0811 05:32:44.010455 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0811 05:32:44.010494 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0811 05:32:44.010531 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0811 05:32:44.010570 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0811 05:32:44.010610 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0811 05:32:44.010649 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0811 05:32:44.010687 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0811 05:32:44.010723 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0811 05:32:44.010762 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0811 05:32:44.010801 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0811 05:32:44.012414 40506 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0811 05:32:44.012447 40506 keypairs.go:95] DescribeKeyPairs: { } I0811 05:32:44.012489 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0811 05:32:44.012555 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0811 05:32:44.012635 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0811 05:32:44.012697 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0811 05:32:44.012744 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0811 05:32:44.012789 40506 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0811 05:32:44.012833 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0811 05:32:44.012871 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0811 05:32:44.012912 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0811 05:32:44.012959 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0811 05:32:44.012996 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } W0811 05:32:44.013045 40506 api.go:53] PageSize not implemented I0811 05:32:44.013063 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I0811 05:32:44.013117 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0811 05:32:44.013148 40506 loadbalancers.go:34] PageSize not implemented I0811 05:32:44.013168 40506 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0811 05:32:44.013193 40506 targetgroups.go:35] PageSize not implemented I0811 05:32:44.013213 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0811 05:32:44.013263 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:44.013290 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:44.013338 40506 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0811 05:32:44.013381 40506 iamrole.go:89] ListRoles: { } I0811 05:32:44.013426 40506 internetgateways.go:101] DescribeInternetGateways: { } I0811 05:32:44.013470 40506 routetable.go:53] DescribeRouteTables: { } I0811 05:32:44.013981 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0811 05:32:44.014063 40506 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } load-balancer:bastion-private-shared-su-5ol32q ok I0811 05:32:44.014114 40506 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } autoscaling-group:nodes.private-shared-subnet.example.com ok I0811 05:32:44.014207 40506 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } iam-instance-profile:masters.private-shared-subnet.example.com ok autoscaling-group:master-us-test-1a.masters.private-shared-subnet.example.com ok I0811 05:32:44.014051 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W0811 05:32:44.014261 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:44.014282 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0811 05:32:44.014217 40506 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } load-balancer:api-private-shared-subnet-n2f8ak ok I0811 05:32:44.014293 40506 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0811 05:32:44.014333 40506 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } autoscaling-group:bastion.private-shared-subnet.example.com ok I0811 05:32:44.014351 40506 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } iam-instance-profile:bastions.private-shared-subnet.example.com ok I0811 05:32:44.014383 40506 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-1 ok I0811 05:32:44.014414 40506 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0811 05:32:44.014484 40506 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } I0811 05:32:44.014472 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } iam-instance-profile:nodes.private-shared-subnet.example.com ok volume:vol-1 ok I0811 05:32:44.014528 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } W0811 05:32:44.014645 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:44.014667 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0811 05:32:44.014505 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } autoscaling-config:lt-2 ok route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.private-shared-subnet.example.com. ok I0811 05:32:44.014737 40506 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0811 05:32:44.014794 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:44.014844 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0811 05:32:44.014925 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:44.014940 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-3 ok I0811 05:32:44.015038 40506 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-subnet.example.com" } I0811 05:32:44.015089 40506 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0811 05:32:44.015063 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:44.015106 40506 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0811 05:32:44.015170 40506 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-subnet.example.com" } iam-role:bastions.private-shared-subnet.example.com ok I0811 05:32:44.015150 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0811 05:32:44.015214 40506 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-subnet.example.com" } I0811 05:32:44.015266 40506 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0811 05:32:44.015283 40506 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0811 05:32:44.015315 40506 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-subnet.example.com" } iam-role:nodes.private-shared-subnet.example.com ok I0811 05:32:44.015352 40506 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-subnet.example.com" } I0811 05:32:44.015397 40506 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0811 05:32:44.015410 40506 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0811 05:32:44.015440 40506 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-subnet.example.com" } iam-role:masters.private-shared-subnet.example.com ok I0811 05:32:44.015248 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W0811 05:32:44.015478 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:44.015503 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0811 05:32:44.015543 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0811 05:32:44.015664 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:44.015695 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0811 05:32:44.015975 40506 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedSubnet (0.36s) === RUN TestLifecyclePrivateSharedIP I0811 05:32:44.016255 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:44.016499 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:44.016547 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:44.016591 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:44.016641 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:44.016686 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:44.016720 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:44.016759 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:44.016786 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:44.016841 40506 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-ip.example.com I0811 05:32:44.051192 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:44.054330 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:44.054475 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:44.054576 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:44.091792 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:44.197363 40506 iamrole.go:89] ListRoles: { } I0811 05:32:44.199660 40506 executor.go:111] Tasks: 0 done / 113 total; 45 can run I0811 05:32:44.200289 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } I0811 05:32:44.200430 40506 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0811 05:32:44.200395 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0811 05:32:44.200640 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } I0811 05:32:44.200723 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:44.200741 40506 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0811 05:32:44.200707 40506 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0811 05:32:44.200924 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0811 05:32:44.200986 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } I0811 05:32:44.201003 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:44.201073 40506 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-ip.example.com" }] } I0811 05:32:44.201118 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:44.201257 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0811 05:32:44.201479 40506 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-ip.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0811 05:32:44.201642 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0811 05:32:44.201775 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0811 05:32:44.201994 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0811 05:32:44.202076 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0811 05:32:44.202118 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0811 05:32:44.202593 40506 executor.go:111] Tasks: 45 done / 113 total; 26 can run I0811 05:32:44.202744 40506 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0811 05:32:44.202821 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0811 05:32:44.202718 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I0811 05:32:44.202896 40506 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0811 05:32:44.202915 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0811 05:32:44.203018 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0811 05:32:44.203039 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0811 05:32:44.203134 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-12345678" } I0811 05:32:44.203250 40506 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0811 05:32:44.203233 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0811 05:32:44.203341 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0811 05:32:44.203450 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0811 05:32:44.203477 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0811 05:32:44.203561 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0811 05:32:44.203619 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:44.203605 40506 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0811 05:32:44.203663 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0811 05:32:44.203753 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0811 05:32:44.203798 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:44.203843 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0811 05:32:44.203962 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0811 05:32:44.204031 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0811 05:32:44.204184 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0811 05:32:44.204091 40506 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] }], VpcId: "vpc-12345678" } I0811 05:32:44.204267 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0811 05:32:44.204317 40506 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] } I0811 05:32:44.204322 40506 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-12345678" } I0811 05:32:44.204393 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0811 05:32:44.204521 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0811 05:32:44.204614 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0811 05:32:44.204722 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0811 05:32:44.204826 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0811 05:32:44.204925 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0811 05:32:44.204919 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], VpcId: "vpc-12345678" } I0811 05:32:44.204972 40506 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] } I0811 05:32:44.205090 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0811 05:32:44.205160 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0811 05:32:44.205204 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I0811 05:32:44.205307 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0811 05:32:44.205360 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0811 05:32:44.205405 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:32:44.205459 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:32:44.205499 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I0811 05:32:44.205610 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0811 05:32:44.205654 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0811 05:32:44.205727 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0811 05:32:44.205776 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0811 05:32:44.205403 40506 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0811 05:32:44.207767 40506 executor.go:111] Tasks: 71 done / 113 total; 36 can run I0811 05:32:44.207971 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:44.208098 40506 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } W0811 05:32:44.208164 40506 api.go:53] PageSize not implemented I0811 05:32:44.208198 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } W0811 05:32:44.208241 40506 api.go:53] PageSize not implemented I0811 05:32:44.208370 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0811 05:32:44.208335 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0811 05:32:44.208491 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0811 05:32:44.208590 40506 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] } I0811 05:32:44.208656 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0811 05:32:44.208591 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0811 05:32:44.208696 40506 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0811 05:32:44.208753 40506 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0811 05:32:44.208753 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-ip.example.com" }] }] } I0811 05:32:44.208835 40506 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"], Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "api.private-shared-ip.example.com" }] } I0811 05:32:44.208899 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0811 05:32:44.208933 40506 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0811 05:32:44.208897 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0811 05:32:44.208980 40506 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0811 05:32:44.209131 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-ingress-tcp-22to22-bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0811 05:32:44.209330 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:44.209514 40506 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0811 05:32:44.209609 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:44.209701 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } I0811 05:32:44.209841 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0811 05:32:44.209912 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0811 05:32:44.210053 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:44.210187 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:44.210316 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0811 05:32:44.210366 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:44.210471 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0811 05:32:44.210643 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-2382to4000-masters.private-shared-ip.example.com" }] }] } I0811 05:32:44.210821 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0811 05:32:44.210989 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0811 05:32:44.211124 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0811 05:32:44.211298 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0811 05:32:44.211451 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0811 05:32:44.211622 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I0811 05:32:44.211756 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0811 05:32:44.211816 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0811 05:32:44.211966 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0811 05:32:44.212012 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0811 05:32:44.212177 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:44.212235 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0811 05:32:44.212403 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0811 05:32:44.212578 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0811 05:32:44.212744 40506 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0811 05:32:44.212801 40506 tags.go:42] CreateTags { Resources: ["eipalloc-12345678"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "shared" }] } I0811 05:32:44.212857 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-4003to65535-masters.private-shared-ip.example.com" }] }] } I0811 05:32:44.213022 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:44.213122 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-udp-1to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0811 05:32:44.213292 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:44.213344 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-1to2379-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0811 05:32:44.213507 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0811 05:32:44.213663 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0811 05:32:44.213706 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:44.214810 40506 executor.go:111] Tasks: 107 done / 113 total; 5 can run I0811 05:32:44.214936 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:44.215005 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:44.215110 40506 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:32:44.215107 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:44.215249 40506 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-12345678", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0811 05:32:44.215344 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0811 05:32:44.215388 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:32:44.215431 40506 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0811 05:32:44.215510 40506 executor.go:111] Tasks: 112 done / 113 total; 1 can run I0811 05:32:44.215574 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:44.215729 40506 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0811 05:32:44.215806 40506 executor.go:111] Tasks: 113 done / 113 total; 0 can run I0811 05:32:44.215849 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:44.215882 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:44.215934 40506 dns.go:238] Pre-creating DNS records I0811 05:32:44.215960 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:44.222219 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0811 05:32:44.225212 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:44.225308 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:44.225375 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:44.262865 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:44.363354 40506 iamrole.go:89] ListRoles: { } I0811 05:32:44.365466 40506 executor.go:111] Tasks: 0 done / 113 total; 45 can run I0811 05:32:44.365801 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0811 05:32:44.365894 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:44.365963 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0811 05:32:44.366081 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0811 05:32:44.366181 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:44.366296 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:44.366450 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0811 05:32:44.366610 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0811 05:32:44.368545 40506 executor.go:111] Tasks: 45 done / 113 total; 26 can run I0811 05:32:44.368762 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0811 05:32:44.368853 40506 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0811 05:32:44.368719 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0811 05:32:44.368931 40506 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0811 05:32:44.368958 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] }] } I0811 05:32:44.369195 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0811 05:32:44.369260 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0811 05:32:44.369372 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0811 05:32:44.369506 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:44.369552 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0811 05:32:44.369673 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0811 05:32:44.369708 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0811 05:32:44.369838 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0811 05:32:44.374537 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0811 05:32:44.375137 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0811 05:32:44.375408 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0811 05:32:44.375459 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I0811 05:32:44.375593 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0811 05:32:44.375700 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:32:44.375752 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:44.378799 40506 executor.go:111] Tasks: 71 done / 113 total; 36 can run I0811 05:32:44.379039 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0811 05:32:44.379184 40506 api.go:53] PageSize not implemented I0811 05:32:44.379180 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:44.379214 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0811 05:32:44.379323 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0811 05:32:44.379365 40506 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0811 05:32:44.379298 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } W0811 05:32:44.379403 40506 api.go:53] PageSize not implemented I0811 05:32:44.379458 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0811 05:32:44.379504 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:44.379568 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0811 05:32:44.379607 40506 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0811 05:32:44.379750 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:44.379997 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:44.380531 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:44.380801 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:44.380904 40506 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0811 05:32:44.380996 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:44.381133 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0811 05:32:44.381315 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0811 05:32:44.382221 40506 executor.go:111] Tasks: 107 done / 113 total; 5 can run W0811 05:32:44.382340 40506 api.go:53] PageSize not implemented I0811 05:32:44.382372 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0811 05:32:44.382396 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0811 05:32:44.382477 40506 dnsname.go:121] AliasTarget for "api.private-shared-ip.example.com" is "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com" I0811 05:32:44.382463 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:44.382568 40506 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0811 05:32:44.382510 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } W0811 05:32:44.382685 40506 api.go:53] PageSize not implemented I0811 05:32:44.382704 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0811 05:32:44.382883 40506 executor.go:111] Tasks: 112 done / 113 total; 1 can run I0811 05:32:44.382980 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0811 05:32:44.383155 40506 executor.go:111] Tasks: 113 done / 113 total; 0 can run I0811 05:32:44.386662 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0811 05:32:44.386724 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0811 05:32:44.386765 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0811 05:32:44.386799 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0811 05:32:44.386837 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0811 05:32:44.386871 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0811 05:32:44.386906 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0811 05:32:44.386945 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0811 05:32:44.386986 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0811 05:32:44.387032 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0811 05:32:44.387072 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0811 05:32:44.387110 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0811 05:32:44.387147 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0811 05:32:44.387179 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0811 05:32:44.387219 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0811 05:32:44.387254 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0811 05:32:44.387291 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0811 05:32:44.387328 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0811 05:32:44.387364 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0811 05:32:44.387399 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0811 05:32:44.387435 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0811 05:32:44.387471 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:32:44.387508 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0811 05:32:44.387543 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0811 05:32:44.387582 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0811 05:32:44.387615 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0811 05:32:44.387649 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0811 05:32:44.387685 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0811 05:32:44.387724 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0811 05:32:44.389315 40506 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0811 05:32:44.389346 40506 keypairs.go:95] DescribeKeyPairs: { } I0811 05:32:44.389387 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0811 05:32:44.389454 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0811 05:32:44.389531 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0811 05:32:44.389593 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0811 05:32:44.389635 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0811 05:32:44.389678 40506 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0811 05:32:44.389719 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0811 05:32:44.389766 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0811 05:32:44.389819 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0811 05:32:44.389875 40506 routetable.go:53] DescribeRouteTables: { } I0811 05:32:44.389902 40506 natgateway.go:117] DescribeNatGateways: { } I0811 05:32:44.389944 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0811 05:32:44.389982 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } W0811 05:32:44.390028 40506 api.go:53] PageSize not implemented I0811 05:32:44.390045 40506 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0811 05:32:44.390098 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0811 05:32:44.390128 40506 loadbalancers.go:34] PageSize not implemented I0811 05:32:44.390148 40506 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0811 05:32:44.390173 40506 targetgroups.go:35] PageSize not implemented I0811 05:32:44.390191 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0811 05:32:44.390243 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:44.390271 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:44.390315 40506 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0811 05:32:44.390355 40506 iamrole.go:89] ListRoles: { } I0811 05:32:44.390408 40506 internetgateways.go:101] DescribeInternetGateways: { } I0811 05:32:44.390457 40506 routetable.go:53] DescribeRouteTables: { } I0811 05:32:44.390491 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0811 05:32:44.390529 40506 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0811 05:32:44.390563 40506 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0811 05:32:44.391159 40506 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0811 05:32:44.391222 40506 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } iam-instance-profile:nodes.private-shared-ip.example.com ok autoscaling-group:nodes.private-shared-ip.example.com ok I0811 05:32:44.391209 40506 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0811 05:32:44.391283 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0811 05:32:44.391284 40506 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } load-balancer:api-private-shared-ip-exa-ohatqj ok I0811 05:32:44.391331 40506 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0811 05:32:44.391413 40506 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } I0811 05:32:44.391384 40506 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0811 05:32:44.391338 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } W0811 05:32:44.391474 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions autoscaling-group:master-us-test-1a.masters.private-shared-ip.example.com ok I0811 05:32:44.391497 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } autoscaling-group:bastion.private-shared-ip.example.com ok I0811 05:32:44.391464 40506 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } security-group:sg-3 ok iam-instance-profile:bastions.private-shared-ip.example.com ok I0811 05:32:44.391348 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } autoscaling-config:lt-3 ok I0811 05:32:44.391539 40506 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } iam-instance-profile:masters.private-shared-ip.example.com ok load-balancer:bastion-private-shared-ip-eepmph ok I0811 05:32:44.391570 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } route53-record:Z1AFAKE1ZON3YO/A/api.private-shared-ip.example.com ok I0811 05:32:44.391630 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0811 05:32:44.391727 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:44.391751 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0811 05:32:44.391786 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0811 05:32:44.391843 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W0811 05:32:44.392045 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:44.392068 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0811 05:32:44.392129 40506 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok autoscaling-config:lt-1 ok I0811 05:32:44.392261 40506 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-2 ok I0811 05:32:44.392372 40506 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0811 05:32:44.392488 40506 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-ip.example.com" } I0811 05:32:44.392542 40506 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0811 05:32:44.392559 40506 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0811 05:32:44.392595 40506 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-ip.example.com" } I0811 05:32:44.392550 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } iam-role:masters.private-shared-ip.example.com ok I0811 05:32:44.392624 40506 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-ip.example.com" } I0811 05:32:44.392672 40506 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0811 05:32:44.392685 40506 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0811 05:32:44.392716 40506 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-ip.example.com" } iam-role:nodes.private-shared-ip.example.com ok I0811 05:32:44.392654 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0811 05:32:44.392771 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:44.392756 40506 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-ip.example.com" } I0811 05:32:44.392847 40506 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0811 05:32:44.392867 40506 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0811 05:32:44.392902 40506 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-ip.example.com" } iam-role:bastions.private-shared-ip.example.com ok I0811 05:32:44.392797 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0811 05:32:44.392957 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0811 05:32:44.393029 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W0811 05:32:44.393156 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:44.393178 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0811 05:32:44.393224 40506 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0811 05:32:44.393299 40506 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0811 05:32:44.393395 40506 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0811 05:32:44.393460 40506 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0811 05:32:44.393743 40506 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedIP (0.38s) === RUN TestLifecycleNodeTerminationHandlerQueueProcessor I0811 05:32:44.394075 40506 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:44.394329 40506 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:44.394379 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:44.394424 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:44.394473 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:44.394517 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:44.394545 40506 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:44.394584 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:44.394612 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:44.394665 40506 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster nthsqsresources.longclustername.example.com I0811 05:32:44.418752 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0811 05:32:44.421500 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:44.422018 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:44.460349 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:44.584299 40506 iamrole.go:89] ListRoles: { } I0811 05:32:44.585990 40506 executor.go:111] Tasks: 0 done / 99 total; 45 can run I0811 05:32:44.586178 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:44.586690 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:44.586758 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:44.586896 40506 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" }] }] } I0811 05:32:44.587010 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } I0811 05:32:44.587051 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] }] } I0811 05:32:44.587180 40506 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] } I0811 05:32:44.587262 40506 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0811 05:32:44.588533 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0811 05:32:44.588660 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0811 05:32:44.588817 40506 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } I0811 05:32:44.588993 40506 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I0811 05:32:44.588973 40506 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0811 05:32:44.589508 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] }] } I0811 05:32:44.590094 40506 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0811 05:32:44.589680 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0811 05:32:44.590522 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0811 05:32:44.590701 40506 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0811 05:32:44.590766 40506 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0811 05:32:44.590801 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0811 05:32:44.590866 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:44.590917 40506 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0811 05:32:44.591102 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0811 05:32:44.594246 40506 executor.go:111] Tasks: 45 done / 99 total; 24 can run I0811 05:32:44.594427 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0811 05:32:44.594622 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0811 05:32:44.594780 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0811 05:32:44.594652 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0811 05:32:44.594933 40506 iamrole.go:161] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0811 05:32:44.594943 40506 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0811 05:32:44.594860 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I0811 05:32:44.595009 40506 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0811 05:32:44.595143 40506 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0811 05:32:44.595167 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0811 05:32:44.595469 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:44.595654 40506 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0811 05:32:44.595752 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0811 05:32:44.595965 40506 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0811 05:32:44.596015 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I0811 05:32:44.596260 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:44.596436 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0811 05:32:44.596504 40506 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" } ] } I0811 05:32:44.596589 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0811 05:32:44.596689 40506 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0811 05:32:44.596733 40506 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" }] }] } I0811 05:32:44.596814 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0811 05:32:44.596880 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:32:44.596929 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:32:44.596973 40506 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0811 05:32:44.597068 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0811 05:32:44.597119 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:44.597172 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0811 05:32:44.597227 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:44.597541 40506 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\",\n \"sqs:DeleteMessage\",\n \"sqs:ReceiveMessage\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:CompleteLifecycleAction\",\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0811 05:32:44.600233 40506 executor.go:111] Tasks: 69 done / 99 total; 26 can run I0811 05:32:44.600435 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0811 05:32:44.600673 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:44.600892 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0811 05:32:44.600947 40506 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0811 05:32:44.600995 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0811 05:32:44.601176 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:44.601290 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:44.601411 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:44.601607 40506 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0811 05:32:44.601685 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-1to2379-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0811 05:32:44.601899 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0811 05:32:44.602115 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:44.602242 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:44.602328 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:44.602599 40506 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0811 05:32:44.602743 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0811 05:32:44.602906 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-2382to4000-masters.nthsqsresources.longclustername.example.com" }] }] } I0811 05:32:44.603061 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" }] }] } I0811 05:32:44.603204 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0811 05:32:44.603337 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0811 05:32:44.603489 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" }] }] } I0811 05:32:44.603617 40506 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0811 05:32:44.603832 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-4003to65535-masters.nthsqsresources.longclustername.example.com" }] }] } I0811 05:32:44.603982 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0811 05:32:44.604204 40506 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-udp-1to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0811 05:32:44.604468 40506 executor.go:111] Tasks: 95 done / 99 total; 2 can run I0811 05:32:44.604741 40506 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0811 05:32:44.604957 40506 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0811 05:32:44.605012 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:44.605062 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:44.605119 40506 dns.go:238] Pre-creating DNS records I0811 05:32:44.605148 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:44.611315 40506 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0811 05:32:44.614357 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:44.614450 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:44.651126 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:44.773237 40506 iamrole.go:89] ListRoles: { } I0811 05:32:44.775032 40506 executor.go:111] Tasks: 0 done / 99 total; 45 can run I0811 05:32:44.775330 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] }] } I0811 05:32:44.775744 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0811 05:32:44.775820 40506 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0811 05:32:44.775904 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:44.775962 40506 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0811 05:32:44.776030 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0811 05:32:44.776145 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0811 05:32:44.776173 40506 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0811 05:32:44.776214 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:44.776240 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0811 05:32:44.777659 40506 executor.go:111] Tasks: 45 done / 99 total; 24 can run I0811 05:32:44.777846 40506 iamrole.go:161] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0811 05:32:44.777997 40506 iamrole.go:161] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0811 05:32:44.777938 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I0811 05:32:44.778105 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:44.778208 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0811 05:32:44.778264 40506 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:32:44.778337 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0811 05:32:44.778468 40506 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:32:44.778549 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] }] } I0811 05:32:44.778700 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I0811 05:32:44.778854 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0811 05:32:44.778902 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0811 05:32:44.786714 40506 executor.go:111] Tasks: 69 done / 99 total; 26 can run I0811 05:32:44.787071 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:44.787462 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:44.787650 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:44.787768 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:44.787947 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:44.788180 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:44.788346 40506 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0811 05:32:44.790088 40506 executor.go:111] Tasks: 95 done / 99 total; 2 can run I0811 05:32:44.790295 40506 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0811 05:32:44.790408 40506 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0811 05:32:44.793293 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0811 05:32:44.793355 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0811 05:32:44.793392 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0811 05:32:44.793425 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0811 05:32:44.793466 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0811 05:32:44.793498 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0811 05:32:44.793532 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0811 05:32:44.793564 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0811 05:32:44.793601 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0811 05:32:44.793635 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0811 05:32:44.793675 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0811 05:32:44.793716 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0811 05:32:44.793749 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0811 05:32:44.793781 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0811 05:32:44.793818 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0811 05:32:44.793850 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0811 05:32:44.793887 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0811 05:32:44.793922 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:32:44.793961 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0811 05:32:44.793995 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0811 05:32:44.794027 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0811 05:32:44.794059 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0811 05:32:44.794097 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0811 05:32:44.794135 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0811 05:32:44.794170 40506 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0811 05:32:44.795597 40506 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0811 05:32:44.795625 40506 keypairs.go:95] DescribeKeyPairs: { } I0811 05:32:44.795663 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0811 05:32:44.795736 40506 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I0811 05:32:44.795794 40506 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0811 05:32:44.795852 40506 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0811 05:32:44.795898 40506 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0811 05:32:44.795951 40506 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0811 05:32:44.795993 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0811 05:32:44.796032 40506 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I0811 05:32:44.796077 40506 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0811 05:32:44.796148 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0811 05:32:44.796214 40506 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } W0811 05:32:44.796303 40506 api.go:53] PageSize not implemented I0811 05:32:44.796333 40506 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0811 05:32:44.796378 40506 loadbalancers.go:34] PageSize not implemented I0811 05:32:44.796418 40506 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0811 05:32:44.796459 40506 targetgroups.go:35] PageSize not implemented I0811 05:32:44.796487 40506 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0811 05:32:44.796938 40506 zones.go:93] ListHostedZonesPages { } I0811 05:32:44.796983 40506 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:44.797067 40506 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0811 05:32:44.797109 40506 iamrole.go:89] ListRoles: { } I0811 05:32:44.797157 40506 internetgateways.go:101] DescribeInternetGateways: { } I0811 05:32:44.797199 40506 routetable.go:53] DescribeRouteTables: { } I0811 05:32:44.797232 40506 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0811 05:32:44.797690 40506 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-1 ok I0811 05:32:44.797790 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-group:nodes.nthsqsresources.longclustername.example.com ok I0811 05:32:44.797866 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0811 05:32:44.798001 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:44.797800 40506 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0811 05:32:44.798022 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } autoscaling-group:master-us-test-1a.masters.nthsqsresources.longclustername.example.com ok security-group:sg-2 ok I0811 05:32:44.797896 40506 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0811 05:32:44.798054 40506 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } route53-record:Z1AFAKE1ZON3YO/A/api.internal.nthsqsresources.longclustername.example.com. ok I0811 05:32:44.798051 40506 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } iam-instance-profile:masters.nthsqsresources.longclustername.example.com ok I0811 05:32:44.798186 40506 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0811 05:32:44.798106 40506 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0811 05:32:44.798302 40506 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0811 05:32:44.798318 40506 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-2 ok I0811 05:32:44.798270 40506 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } iam-instance-profile:nodes.nthsqsresources.longclustername.example.com ok I0811 05:32:44.798373 40506 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0811 05:32:44.798436 40506 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0811 05:32:44.798481 40506 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0811 05:32:44.798550 40506 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0811 05:32:44.798641 40506 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0811 05:32:44.798699 40506 iamrole.go:180] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0811 05:32:44.798680 40506 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0811 05:32:44.798718 40506 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.nthsqsresources.longclustername.example.com" } I0811 05:32:44.798817 40506 iamrole.go:180] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0811 05:32:44.798838 40506 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0811 05:32:44.798877 40506 iamrole.go:137] DeleteRole: { RoleName: "masters.nthsqsresources.longclustername.example.com" } iam-role:masters.nthsqsresources.longclustername.example.com ok I0811 05:32:44.798918 40506 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0811 05:32:44.798951 40506 iamrole.go:137] DeleteRole: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } iam-role:nodes.nthsqsresources.longclustername.example.com ok I0811 05:32:44.799051 40506 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0811 05:32:44.799205 40506 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0811 05:32:44.799364 40506 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0811 05:32:44.799841 40506 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleNodeTerminationHandlerQueueProcessor (0.41s) === RUN TestGetFilters --- PASS: TestGetFilters (0.00s) === RUN TestGetInstanceSelectorOpts --- PASS: TestGetInstanceSelectorOpts (0.00s) === RUN TestValidateAllPrivateOrPublicSubnets --- PASS: TestValidateAllPrivateOrPublicSubnets (0.00s) === RUN TestValidateUserSubnetsWithClusterSubnets --- PASS: TestValidateUserSubnetsWithClusterSubnets (0.00s) === RUN TestCreateInstanceGroup --- PASS: TestCreateInstanceGroup (0.00s) === RUN TestDecorateWithInstanceGroupSpecs --- PASS: TestDecorateWithInstanceGroupSpecs (0.00s) === RUN TestDecorateWithMixedInstancesPolicy --- PASS: TestDecorateWithMixedInstancesPolicy (0.00s) === RUN TestDecorateWithClusterAutoscalerLabels --- PASS: TestDecorateWithClusterAutoscalerLabels (0.00s) === RUN TestNewTemplateContext --- PASS: TestNewTemplateContext (0.00s) PASS ok k8s.io/kops/cmd/kops 43.187s ? k8s.io/kops/cmd/kops/util [no test files] ? k8s.io/kops/cmd/kops-controller [no test files] ? k8s.io/kops/cmd/kops-controller/controllers [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/config [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/server [no test files] === RUN TestBuildProxyRequest === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/livez === RUN TestBuildProxyRequest/http://127.0.0.1:3990/healthz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ready === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz? === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude --- PASS: TestBuildProxyRequest (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/livez (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/healthz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ready (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz? (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude (0.00s) PASS ok k8s.io/kops/cmd/kube-apiserver-healthcheck 0.018s ? k8s.io/kops/cmd/nodeup [no test files] ? k8s.io/kops/dns-controller/cmd/dns-controller [no test files] === RUN TestAliasForNodesInRole --- PASS: TestAliasForNodesInRole (0.00s) === RUN TestEnsureDotSuffix --- PASS: TestEnsureDotSuffix (0.00s) === RUN TestParseZoneSpec --- PASS: TestParseZoneSpec (0.00s) === RUN TestParseZoneRules I0811 05:30:57.214642 16307 zonespec.go:75] No rules specified, will permit management of all zones --- PASS: TestParseZoneRules (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/dns 0.059s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/util 0.084s === RUN TestPodController I0811 05:31:31.729186 34685 pod.go:59] starting pod controller I0811 05:31:31.731030 34685 stoppable.go:62] shutting down controller I0811 05:31:31.731120 34685 pod.go:65] shutting down pod controller I0811 05:31:31.731160 34685 pod.go:107] Got stop signal --- PASS: TestPodController (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/watchers 0.022s === RUN TestEquivalent --- PASS: TestEquivalent (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider 0.091s Parsing flags. Getting new test interface. Running tests... === RUN TestZonesList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00010ddc0} route53_test.go:93: Got zone list: [0xc00010ddf0] route53_test.go:98: Got at least 1 zone in list:&{0xc000338fc0 0xc000116080} --- PASS: TestZonesList (0.00s) === RUN TestZonesID route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00010dec0} route53_test.go:93: Got zone list: [0xc00010def0] route53_test.go:98: Got at least 1 zone in list:&{0xc000338fc0 0xc000116090} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess route53_test.go:79: Got zones {0xc00010df90} route53_test.go:170: Successfully added managed DNS zone: &{0xc0003391a0 0xc0001160a8} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00035e0f0} route53_test.go:93: Got zone list: [0xc00035e120] route53_test.go:98: Got at least 1 zone in list:&{0xc000338fc0 0xc0001160c0} route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00035e220} route53_test.go:93: Got zone list: [0xc00035e250] route53_test.go:98: Got at least 1 zone in list:&{0xc000338fc0 0xc0001160e0} route53_test.go:187: Successfully added resource record set: {0xc000182100 0xc000116100} --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00035e4d0} route53_test.go:93: Got zone list: [0xc00035e500] route53_test.go:98: Got at least 1 zone in list:&{0xc000338fc0 0xc000116188} route53_test.go:199: Successfully added resource record set: {0xc000182280 0xc0001161a8} route53_test.go:118: Got 1 recordsets: [0xc00035e6a0] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00035e7d0} route53_test.go:93: Got zone list: [0xc00035e800] route53_test.go:98: Got at least 1 zone in list:&{0xc000338fc0 0xc000116240} route53_test.go:221: Successfully added resource record set: {0xc000182400 0xc000116260} route53_test.go:228: Correctly failed to add duplicate resource record {0xc000182400 0xc000116260}: attempt to create duplicate rrset www11.example.com::A --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00035eb60} route53_test.go:93: Got zone list: [0xc00035eb90] route53_test.go:98: Got at least 1 zone in list:&{0xc000338fc0 0xc000116318} route53_test.go:246: Successfully removed resource set {0xc000182600 0xc000116338} after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00035ee10} route53_test.go:93: Got zone list: [0xc00035ee40] route53_test.go:98: Got at least 1 zone in list:&{0xc000338fc0 0xc0001163c0} route53_test.go:264: Successfully removed resource set {0xc000182780 0xc0001163e0} after adding route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00035f0f0} route53_test.go:93: Got zone list: [0xc00035f120] route53_test.go:98: Got at least 1 zone in list:&{0xc000338fc0 0xc000116470} commontests.go:192: Successfully added resource record set: {0xc000182900 0xc000116490} commontests.go:45: Correctly replaced resource record {0xc000182900 0xc000116490} -> {0xc000182a00 0xc0001164e0} commontests.go:142: Got 1 recordsets: [0xc00035f430] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00035f600} route53_test.go:93: Got zone list: [0xc00035f630] route53_test.go:98: Got at least 1 zone in list:&{0xc000338fc0 0xc000116590} commontests.go:192: Successfully added resource record set: {0xc000182c80 0xc0001165b0} commontests.go:70: Correctly replaced resource record {0xc000182c80 0xc0001165b0} -> {0xc000182d80 0xc000116600} commontests.go:142: Got 1 recordsets: [0xc00035f940] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00035fb60} route53_test.go:93: Got zone list: [0xc00035fb90] route53_test.go:98: Got at least 1 zone in list:&{0xc000338fc0 0xc0001166c0} commontests.go:192: Successfully added resource record set: {0xc000183000 0xc0001166e0} commontests.go:142: Got 2 recordsets: [0xc00035fe00 0xc00035fe20] commontests.go:142: Got 2 recordsets: [0xc00035fe90 0xc00035feb0] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000370050} route53_test.go:93: Got zone list: [0xc000370080] route53_test.go:98: Got at least 1 zone in list:&{0xc000338fc0 0xc0001167e0} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53 0.009s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53/stubs [no test files] === RUN TestZonesList --- PASS: TestZonesList (0.00s) === RUN TestAdd --- PASS: TestAdd (0.00s) === RUN TestRemove --- PASS: TestRemove (0.00s) === RUN TestNewZone --- PASS: TestNewZone (0.00s) === RUN TestNewResourceRecordSet --- PASS: TestNewResourceRecordSet (0.00s) === RUN TestResourceRecordChangeset --- PASS: TestResourceRecordChangeset (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/do 0.018s === RUN TestZonesList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00011b9f0 0xc0005a7940} clouddns_test.go:71: Got zone list: [0xc00000c2a0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00036ba00 0xc00000c270} --- PASS: TestZonesList (0.00s) === RUN TestZonesID clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00011b9f0 0xc0005a7a40} clouddns_test.go:71: Got zone list: [0xc00000c348] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00036ba00 0xc00000c318} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess clouddns_test.go:131: Getting zones clouddns_test.go:58: Got zones {0xc00011b9f0 0xc0005a7b40} clouddns_test.go:133: Got zones, making new Zone clouddns_test.go:149: Successfully added managed DNS zone: &{0xc00036bb40 0xc00000c3f0} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00011b9f0 0xc0005a7c00} clouddns_test.go:71: Got zone list: [0xc00000c498] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00036ba00 0xc00000c468} clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00011b9f0 0xc0005a7d20} clouddns_test.go:71: Got zone list: [0xc00000c5b8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00036ba00 0xc00000c588} clouddns_test.go:165: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00011b9f0 0xc0005a7e60} clouddns_test.go:71: Got zone list: [0xc00000c708] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00036ba00 0xc00000c6d8} clouddns_test.go:176: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:96: Got 1 recordsets: [<(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00011b9f0 0xc0005a7fc0} clouddns_test.go:71: Got zone list: [0xc00000c900] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00036ba00 0xc00000c8d0} clouddns_test.go:198: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:205: Correctly failed to add duplicate resource record <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>: attempt to insert duplicate rrset {www11.example.com [10.10.10.10 169.20.20.20] 180 A} --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00011b9f0 0xc0001d6240} clouddns_test.go:71: Got zone list: [0xc00000cbd0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00036ba00 0xc00000ca50} clouddns_test.go:223: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00011b9f0 0xc0001d6380} clouddns_test.go:71: Got zone list: [0xc00000cd80] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00036ba00 0xc00000cd50} clouddns_test.go:241: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00011b9f0 0xc0001d65c0} clouddns_test.go:71: Got zone list: [0xc00000cf30] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00036ba00 0xc00000cf00} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:45: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00011b9f0 0xc0001d6860} clouddns_test.go:71: Got zone list: [0xc00000d1e8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00036ba00 0xc00000d1b8} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:70: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00011b9f0 0xc0001d6a40} clouddns_test.go:71: Got zone list: [0xc00000d500] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00036ba00 0xc00000d4d0} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00011b9f0 0xc0001d6c60} clouddns_test.go:71: Got zone list: [0xc00000d8c0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00036ba00 0xc00000d890} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns 0.044s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/interfaces [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/stubs [no test files] === RUN TestContract --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/openstack/designate 0.160s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/rrstype [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/tests [no test files] ? k8s.io/kops/examples/kops-api-example [no test files] === RUN TestBootstarapBuilder_Simple --- PASS: TestBootstarapBuilder_Simple (0.00s) PASS ok k8s.io/kops/nodeup/pkg/bootstrap 0.470s === RUN TestBuildAzure --- PASS: TestBuildAzure (0.00s) === RUN TestBuildAWSCustomNodeIPFamilies --- PASS: TestBuildAWSCustomNodeIPFamilies (0.00s) === RUN TestContainerdBuilder_Docker_19_03_13 I0811 05:32:02.082461 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.082675 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.082719 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.082747 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.082786 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.082828 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.082839 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.082871 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.082890 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.090793 40536 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_13 (0.01s) === RUN TestContainerdBuilder_Docker_19_03_14 I0811 05:32:02.094475 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.094635 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.094665 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.094692 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.094720 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.094742 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.094760 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.094782 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.094798 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.095935 40536 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_14 (0.00s) === RUN TestContainerdBuilder_Simple I0811 05:32:02.099245 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.099370 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.099394 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.099417 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.099441 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.099460 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.099488 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.099511 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.099524 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.100435 40536 zones.go:93] ListHostedZonesPages { } I0811 05:32:02.101802 40536 assetstore.go:108] Matching assets for "^bin/(containerd|ctr)": I0811 05:32:02.101819 40536 assetstore.go:111] bin/containerd I0811 05:32:02.101828 40536 assetstore.go:111] bin/containerd-shim I0811 05:32:02.101836 40536 assetstore.go:111] bin/containerd-shim-runc-v1 I0811 05:32:02.101843 40536 assetstore.go:111] bin/containerd-shim-runc-v2 I0811 05:32:02.101849 40536 assetstore.go:111] bin/containerd-stress I0811 05:32:02.101856 40536 assetstore.go:111] bin/ctr I0811 05:32:02.101895 40536 assetstore.go:108] Matching assets for "/runc\\.(amd64|arm64)$": I0811 05:32:02.101909 40536 assetstore.go:111] https://github.com/opencontainers/runc/releases/download/v1.1.0/runc.amd64 --- PASS: TestContainerdBuilder_Simple (0.01s) === RUN TestContainerdBuilder_Flatcar I0811 05:32:02.104876 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.104997 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.105023 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.105046 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.105070 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.105091 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.105109 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.105131 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.105145 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.106102 40536 zones.go:93] ListHostedZonesPages { } I0811 05:32:02.107349 40536 containerd.go:58] Detected Flatcar; won't install containerd --- PASS: TestContainerdBuilder_Flatcar (0.00s) === RUN TestContainerdBuilder_SkipInstall I0811 05:32:02.108201 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.108332 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.108358 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.108381 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.108404 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.108471 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.108491 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.108511 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.108525 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.109401 40536 zones.go:93] ListHostedZonesPages { } I0811 05:32:02.110738 40536 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestContainerdBuilder_SkipInstall (0.00s) === RUN TestContainerdBuilder_BuildFlags --- PASS: TestContainerdBuilder_BuildFlags (0.00s) === RUN TestContainerdConfig --- PASS: TestContainerdConfig (0.00s) === RUN TestAppendGPURuntimeContainerdConfig --- PASS: TestAppendGPURuntimeContainerdConfig (0.00s) === RUN TestDockerBuilder_19_03_11 I0811 05:32:02.111293 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.111413 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.111435 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.111457 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.111482 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.111519 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.111536 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.111554 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.111568 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.113078 40536 zones.go:93] ListHostedZonesPages { } I0811 05:32:02.114441 40536 assetstore.go:108] Matching assets for "^docker/": I0811 05:32:02.114458 40536 assetstore.go:111] docker/containerd I0811 05:32:02.114466 40536 assetstore.go:111] docker/containerd-shim I0811 05:32:02.114473 40536 assetstore.go:111] docker/ctr I0811 05:32:02.114480 40536 assetstore.go:111] docker/docker I0811 05:32:02.114487 40536 assetstore.go:111] docker/docker-init I0811 05:32:02.114495 40536 assetstore.go:111] docker/docker-proxy I0811 05:32:02.114501 40536 assetstore.go:111] docker/dockerd I0811 05:32:02.114509 40536 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_19_03_11 (0.01s) === RUN TestDockerBuilder_LogFlags I0811 05:32:02.117081 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.117204 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.117227 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.117249 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.117271 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.117309 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.117334 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.117356 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.117368 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.118230 40536 zones.go:93] ListHostedZonesPages { } I0811 05:32:02.119640 40536 assetstore.go:108] Matching assets for "^docker/": I0811 05:32:02.119658 40536 assetstore.go:111] docker/containerd I0811 05:32:02.119667 40536 assetstore.go:111] docker/containerd-shim I0811 05:32:02.119674 40536 assetstore.go:111] docker/ctr I0811 05:32:02.119681 40536 assetstore.go:111] docker/docker I0811 05:32:02.119689 40536 assetstore.go:111] docker/docker-init I0811 05:32:02.119696 40536 assetstore.go:111] docker/docker-proxy I0811 05:32:02.119703 40536 assetstore.go:111] docker/dockerd I0811 05:32:02.119710 40536 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_LogFlags (0.01s) === RUN TestDockerBuilder_SkipInstall I0811 05:32:02.122158 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.122272 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.122294 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.122315 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.122338 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.122388 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.122405 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.122435 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.122448 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.124898 40536 zones.go:93] ListHostedZonesPages { } I0811 05:32:02.127574 40536 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestDockerBuilder_SkipInstall (0.01s) === RUN TestDockerBuilder_HealthCheck I0811 05:32:02.127909 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.128159 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.128209 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.128257 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.128315 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.128401 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.128437 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.128476 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.128503 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.130151 40536 zones.go:93] ListHostedZonesPages { } I0811 05:32:02.132550 40536 assetstore.go:108] Matching assets for "^docker/": I0811 05:32:02.132582 40536 assetstore.go:111] docker/docker I0811 05:32:02.132599 40536 assetstore.go:111] docker/docker-containerd I0811 05:32:02.132612 40536 assetstore.go:111] docker/docker-containerd-ctr I0811 05:32:02.132624 40536 assetstore.go:111] docker/docker-containerd-shim I0811 05:32:02.132636 40536 assetstore.go:111] docker/docker-init I0811 05:32:02.132648 40536 assetstore.go:111] docker/docker-proxy I0811 05:32:02.132660 40536 assetstore.go:111] docker/docker-runc I0811 05:32:02.132672 40536 assetstore.go:111] docker/dockerd --- PASS: TestDockerBuilder_HealthCheck (0.01s) === RUN TestDockerBuilder_BuildFlags --- PASS: TestDockerBuilder_BuildFlags (0.00s) === RUN TestContainerdHooksBuilder I0811 05:32:02.139341 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.139546 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.139589 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.139628 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.139669 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.139704 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.139733 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.139766 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.139790 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.143187 40536 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdHooksBuilder (0.01s) === RUN TestDockerHooksBuilder I0811 05:32:02.146363 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.146539 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.146575 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.146609 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.146645 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.146675 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.146703 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.146732 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.146752 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.149080 40536 zones.go:93] ListHostedZonesPages { } --- PASS: TestDockerHooksBuilder (0.01s) === RUN TestKopsControllerBuilder I0811 05:32:02.152289 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.152484 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.152519 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.152554 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.152591 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.152620 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.152644 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.152670 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.152690 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.154627 40536 zones.go:93] ListHostedZonesPages { } --- PASS: TestKopsControllerBuilder (0.01s) === RUN Test_KubeAPIServer_BuildFlags --- PASS: Test_KubeAPIServer_BuildFlags (0.00s) === RUN TestKubeAPIServerBuilder I0811 05:32:02.161637 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.161790 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.161820 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.161846 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.161873 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.161897 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.161917 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.161943 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.161958 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.163707 40536 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilder (0.01s) === RUN TestDeddicatedAPIServerBuilder I0811 05:32:02.173509 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.174062 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.174175 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.174361 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.174471 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.174513 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.174551 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.174625 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.174653 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.176955 40536 zones.go:93] ListHostedZonesPages { } --- PASS: TestDeddicatedAPIServerBuilder (0.01s) === RUN TestWithoutEtcdEventsAPIServerBuilder I0811 05:32:02.182658 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.182783 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.182809 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.182833 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.182858 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.182887 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.182907 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.182926 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.182941 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.184279 40536 zones.go:93] ListHostedZonesPages { } --- PASS: TestWithoutEtcdEventsAPIServerBuilder (0.01s) === RUN TestAwsIamAuthenticator I0811 05:32:02.189950 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.190067 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.190091 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.190115 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.190137 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.190156 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.190173 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.190197 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.190211 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.191690 40536 zones.go:93] ListHostedZonesPages { } --- PASS: TestAwsIamAuthenticator (0.01s) === RUN TestKubeAPIServerBuilderAMD64 I0811 05:32:02.199607 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:02.199873 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:02.199903 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:02.199931 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:02.199956 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:02.199976 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:02.199992 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:02.200012 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:02.200025 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:02.201707 40536 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderAMD64 (1.40s) === RUN TestKubeAPIServerBuilderARM64 I0811 05:32:03.595379 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:03.595640 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:03.595714 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:03.595771 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:03.595821 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:03.595864 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:03.595899 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:03.595940 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:03.595967 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:03.598795 40536 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderARM64 (0.08s) === RUN TestKubeControllerManagerBuilder I0811 05:32:03.670863 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:03.671560 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:03.671617 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:03.671664 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:03.671717 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:03.671760 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:03.671791 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:03.671831 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:03.671863 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:03.674814 40536 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilder (0.01s) === RUN TestKubeControllerManagerBuilderAMD64 I0811 05:32:03.684881 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:03.685139 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:03.685186 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:03.685228 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:03.685253 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:03.685284 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:03.685303 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:03.685332 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:03.685366 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:03.688644 40536 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderAMD64 (0.07s) === RUN TestKubeControllerManagerBuilderARM64 I0811 05:32:03.759820 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:03.759997 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:03.760036 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:03.760073 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:03.760107 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:03.760135 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:03.760153 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:03.760177 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:03.760197 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:03.761890 40536 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderARM64 (0.07s) === RUN TestKubeProxyBuilder_buildPod === RUN TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 --- PASS: TestKubeProxyBuilder_buildPod (0.00s) --- PASS: TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 (0.00s) === RUN TestKubeProxyBuilder I0811 05:32:03.834589 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:03.834868 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:03.834918 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:03.834973 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:03.835033 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:03.835086 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:03.835126 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:03.835188 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:03.835215 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:03.837116 40536 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilder (0.01s) === RUN TestKubeProxyBuilderAMD64 I0811 05:32:03.840665 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:03.840803 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:03.840828 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:03.840852 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:03.840885 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:03.840909 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:03.840926 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:03.840949 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:03.840968 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:03.842470 40536 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderAMD64 (0.07s) === RUN TestKubeProxyBuilderARM64 I0811 05:32:03.912959 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:03.913349 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:03.913434 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:03.913507 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:03.913675 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:03.914226 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:03.914251 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:03.914278 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:03.914296 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:03.916229 40536 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderARM64 (0.07s) === RUN TestParseDefault --- PASS: TestParseDefault (0.00s) === RUN TestParse --- PASS: TestParse (0.00s) === RUN TestKubeSchedulerBuilder I0811 05:32:03.983037 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:03.983219 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:03.983256 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:03.983287 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:03.983322 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:03.983351 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:03.983369 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:03.983394 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:03.983412 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:03.985000 40536 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeSchedulerBuilder (0.01s) === RUN TestKubeSchedulerBuilderAMD64 I0811 05:32:03.989232 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:03.989359 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:03.989387 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:03.989410 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:03.989434 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:03.989457 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:03.989474 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:03.989496 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:03.989509 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:03.991073 40536 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeSchedulerBuilderAMD64 (0.07s) === RUN TestKubeSchedulerBuilderARM64 I0811 05:32:04.058874 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:04.059053 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:04.059085 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:04.059113 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:04.059142 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:04.059167 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:04.059184 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:04.059205 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:04.059221 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:04.060796 40536 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeSchedulerBuilderARM64 (0.07s) === RUN TestKubectlBuilder I0811 05:32:04.127733 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:04.127984 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:04.128055 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:04.128131 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:04.128186 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:04.128242 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:04.128276 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:04.128327 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:04.128361 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:04.131153 40536 zones.go:93] ListHostedZonesPages { } I0811 05:32:04.133012 40536 assetstore.go:159] Resolved asset kubectl: to /path/to/kubectl/asset --- PASS: TestKubectlBuilder (0.01s) === RUN Test_InstanceGroupKubeletMerge --- PASS: Test_InstanceGroupKubeletMerge (0.00s) === RUN TestTaintsApplied --- PASS: TestTaintsApplied (0.00s) === RUN Test_RunKubeletBuilder I0811 05:32:04.134624 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:04.134779 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:04.134831 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:04.134871 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:04.134911 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:04.134942 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:04.134973 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:04.135010 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:04.135031 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:04.136367 40536 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilder (0.01s) === RUN Test_RunKubeletBuilderWarmPool I0811 05:32:04.139845 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:04.139970 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:04.140015 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:04.140055 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:04.140098 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:04.140128 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:04.140158 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:04.140194 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:04.140218 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:04.141437 40536 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilderWarmPool (0.00s) === RUN Test_BuildComponentConfigFile --- PASS: Test_BuildComponentConfigFile (0.00s) === RUN TestProtokubeBuilder I0811 05:32:04.144777 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:04.144921 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:04.144957 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:04.144995 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:04.145037 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:04.145068 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:04.145099 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:04.145134 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:04.145156 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:04.146726 40536 zones.go:93] ListHostedZonesPages { } I0811 05:32:04.148295 40536 assetstore.go:108] Matching assets for "protokube$": I0811 05:32:04.148317 40536 assetstore.go:111] /opt/kops/bin/protokube I0811 05:32:04.148332 40536 assetstore.go:129] Found single matching asset for expr "protokube$": "protokube" I0811 05:32:04.148369 40536 assetstore.go:108] Matching assets for "channels$": I0811 05:32:04.148388 40536 assetstore.go:111] /opt/kops/bin/channels I0811 05:32:04.148400 40536 assetstore.go:129] Found single matching asset for expr "channels$": "channels" --- PASS: TestProtokubeBuilder (0.00s) === RUN TestSecretBuilder I0811 05:32:04.149752 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:04.149890 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:04.149923 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:04.149960 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:04.150002 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:04.150031 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:04.150062 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:04.150097 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:04.150118 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:04.151598 40536 zones.go:93] ListHostedZonesPages { } --- PASS: TestSecretBuilder (0.00s) === RUN TestUpdateServiceBuilderAutomaticUpgrade I0811 05:32:04.153798 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:04.153939 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:04.153974 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:04.154013 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:04.154054 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:04.154087 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:04.154118 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:04.154152 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:04.154178 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:04.155651 40536 zones.go:93] ListHostedZonesPages { } I0811 05:32:04.157187 40536 update_service.go:96] Detected OS {deb ubuntu focal 20.04}; installing unattended-upgrades package --- PASS: TestUpdateServiceBuilderAutomaticUpgrade (0.00s) === RUN TestUpdateServiceBuilderExternal I0811 05:32:04.157602 40536 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:04.157745 40536 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:04.157779 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:04.157821 40536 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:04.157863 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:04.157893 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:04.157926 40536 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:04.157962 40536 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:04.157988 40536 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:04.159477 40536 zones.go:93] ListHostedZonesPages { } I0811 05:32:04.161017 40536 update_service.go:91] UpdatePolicy requests automatic updates; skipping installation of package "unattended-upgrades" --- PASS: TestUpdateServiceBuilderExternal (0.00s) PASS ok k8s.io/kops/nodeup/pkg/model 2.133s ? k8s.io/kops/nodeup/pkg/model/dns [no test files] ? k8s.io/kops/nodeup/pkg/model/networking [no test files] ? k8s.io/kops/nodeup/pkg/model/resources [no test files] ? k8s.io/kops/pkg/acls [no test files] ? k8s.io/kops/pkg/acls/gce [no test files] === RUN Test_Strategy --- PASS: Test_Strategy (0.00s) === RUN Test_In_StateStore --- PASS: Test_In_StateStore (0.00s) PASS ok k8s.io/kops/pkg/acls/s3 0.054s === RUN TestWarmPoolSpec_IsEnabled === RUN TestWarmPoolSpec_IsEnabled/nil === RUN TestWarmPoolSpec_IsEnabled/empty === RUN TestWarmPoolSpec_IsEnabled/1 === RUN TestWarmPoolSpec_IsEnabled/0 --- PASS: TestWarmPoolSpec_IsEnabled (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/nil (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/empty (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/1 (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/0 (0.00s) === RUN TestWarmPoolSpec_ResolveDefaults === RUN TestWarmPoolSpec_ResolveDefaults/MinSize === RUN TestWarmPoolSpec_ResolveDefaults/MaxSize === RUN TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook --- PASS: TestWarmPoolSpec_ResolveDefaults (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MinSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MaxSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook (0.00s) === RUN Test_ParseInstanceGroupRole --- PASS: Test_ParseInstanceGroupRole (0.00s) === RUN TestParseConfigYAML === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{}" --- PASS: TestParseConfigYAML (0.01s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{}" (0.00s) === RUN TestWeaveParseConfigYAML === RUN TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" === RUN TestWeaveParseConfigYAML/"networking:_{}" --- PASS: TestWeaveParseConfigYAML (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{}" (0.00s) === RUN Test_SemverOrdering --- PASS: Test_SemverOrdering (0.00s) PASS ok k8s.io/kops/pkg/apis/kops 0.083s === RUN TestRoundTripTypes --- PASS: TestRoundTripTypes (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/install 0.063s === RUN TestUseCiliumEtcd --- PASS: TestUseCiliumEtcd (0.00s) === RUN Test_FindSubnet --- PASS: Test_FindSubnet (0.00s) === RUN Test_FindZonesForInstanceGroup --- PASS: Test_FindZonesForInstanceGroup (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/model 0.086s ? k8s.io/kops/pkg/apis/kops/registry [no test files] === RUN TestGetNodeRole === RUN TestGetNodeRole/RoleNone === RUN TestGetNodeRole/RoleNewerLabel === RUN TestGetNodeRole/RoleOlderLabel --- PASS: TestGetNodeRole (0.00s) --- PASS: TestGetNodeRole/RoleNone (0.00s) --- PASS: TestGetNodeRole/RoleNewerLabel (0.00s) --- PASS: TestGetNodeRole/RoleOlderLabel (0.00s) === RUN Test_ParseKubernetesVersion === RUN Test_ParseKubernetesVersion/1.3.7 === RUN Test_ParseKubernetesVersion/v1.4.0-beta.8 === RUN Test_ParseKubernetesVersion/1.5.0 === RUN Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ === RUN Test_ParseKubernetesVersion/#00 E0811 05:31:16.323387 26831 versions.go:96] unable to parse Kubernetes version "" === RUN Test_ParseKubernetesVersion/abc E0811 05:31:16.323477 26831 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: Test_ParseKubernetesVersion (0.00s) --- PASS: Test_ParseKubernetesVersion/1.3.7 (0.00s) --- PASS: Test_ParseKubernetesVersion/v1.4.0-beta.8 (0.00s) --- PASS: Test_ParseKubernetesVersion/1.5.0 (0.00s) --- PASS: Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ (0.00s) --- PASS: Test_ParseKubernetesVersion/#00 (0.00s) --- PASS: Test_ParseKubernetesVersion/abc (0.00s) === RUN Test_IsKubernetesGTEWithPatch --- PASS: Test_IsKubernetesGTEWithPatch (0.00s) === RUN Test_IsKubernetesGTEWithoutPatch --- PASS: Test_IsKubernetesGTEWithoutPatch (0.00s) === RUN Test_IsKubernetesGTEWithPre --- PASS: Test_IsKubernetesGTEWithPre (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/util 0.007s ? k8s.io/kops/pkg/apis/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha3 [no test files] === RUN TestAWSValidateExternalCloudConfig --- PASS: TestAWSValidateExternalCloudConfig (0.00s) === RUN TestValidateInstanceGroupSpec I0811 05:31:51.904005 39817 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0811 05:31:51.904120 39817 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0811 05:31:51.904195 39817 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0811 05:31:51.904242 39817 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0811 05:31:51.904280 39817 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestValidateInstanceGroupSpec (0.00s) === RUN TestMixedInstancePolicies I0811 05:31:51.904560 39817 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0811 05:31:51.904607 39817 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0811 05:31:51.904641 39817 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0811 05:31:51.904679 39817 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0811 05:31:51.904718 39817 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0811 05:31:51.904751 39817 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0811 05:31:51.904788 39817 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0811 05:31:51.904820 39817 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0811 05:31:51.904857 39817 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0811 05:31:51.904892 39817 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0811 05:31:51.904922 39817 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0811 05:31:51.904958 39817 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0811 05:31:51.904992 39817 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0811 05:31:51.905022 39817 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0811 05:31:51.905063 39817 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0811 05:31:51.905111 39817 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0811 05:31:51.905146 39817 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0811 05:31:51.905174 39817 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0811 05:31:51.905203 39817 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestMixedInstancePolicies (0.00s) === RUN TestInstanceMetadataOptions I0811 05:31:51.905375 39817 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } I0811 05:31:51.905455 39817 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } --- PASS: TestInstanceMetadataOptions (0.00s) === RUN TestLoadBalancerSubnets --- PASS: TestLoadBalancerSubnets (0.00s) === RUN TestAWSAuthentication --- PASS: TestAWSAuthentication (0.00s) === RUN TestAWSAdditionalRoutes --- PASS: TestAWSAdditionalRoutes (0.00s) === RUN TestValidEtcdChanges --- PASS: TestValidEtcdChanges (0.00s) === RUN TestValidateInstanceProfile --- PASS: TestValidateInstanceProfile (0.00s) === RUN TestValidMasterInstanceGroup --- PASS: TestValidMasterInstanceGroup (0.00s) === RUN TestValidBootDevice --- PASS: TestValidBootDevice (0.00s) === RUN TestValidNodeLabels --- PASS: TestValidNodeLabels (0.00s) === RUN TestValidateIGCloudLabels --- PASS: TestValidateIGCloudLabels (0.00s) === RUN TestIGCloudLabelIsIGName --- PASS: TestIGCloudLabelIsIGName (0.00s) === RUN TestValidTaints --- PASS: TestValidTaints (0.00s) === RUN TestIGUpdatePolicy === RUN TestIGUpdatePolicy/missing === RUN TestIGUpdatePolicy/automatic === RUN TestIGUpdatePolicy/external === RUN TestIGUpdatePolicy/empty === RUN TestIGUpdatePolicy/unknown --- PASS: TestIGUpdatePolicy (0.00s) --- PASS: TestIGUpdatePolicy/missing (0.00s) --- PASS: TestIGUpdatePolicy/automatic (0.00s) --- PASS: TestIGUpdatePolicy/external (0.00s) --- PASS: TestIGUpdatePolicy/empty (0.00s) --- PASS: TestIGUpdatePolicy/unknown (0.00s) === RUN TestValidInstanceGroup --- PASS: TestValidInstanceGroup (0.00s) === RUN Test_ValidateTopology --- PASS: Test_ValidateTopology (0.00s) === RUN Test_Validate_DNS --- PASS: Test_Validate_DNS (0.00s) === RUN TestValidateCIDR --- PASS: TestValidateCIDR (0.00s) === RUN TestValidateSubnets --- PASS: TestValidateSubnets (0.00s) === RUN TestValidateKubeAPIServer --- PASS: TestValidateKubeAPIServer (0.00s) === RUN Test_Validate_DockerConfig_Storage --- PASS: Test_Validate_DockerConfig_Storage (0.00s) === RUN Test_Validate_Networking_Flannel --- PASS: Test_Validate_Networking_Flannel (0.00s) === RUN Test_Validate_AdditionalPolicies --- PASS: Test_Validate_AdditionalPolicies (0.00s) === RUN Test_Validate_Calico === RUN Test_Validate_Calico/empty_specs === RUN Test_Validate_Calico/positive_Typha_replica_count === RUN Test_Validate_Calico/negative_Typha_replica_count === RUN Test_Validate_Calico/with_etcd_version === RUN Test_Validate_Calico/IPv4_autodetection_method === RUN Test_Validate_Calico/IPv6_autodetection_method === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name === RUN Test_Validate_Calico/AWS_source/destination_checks_off === RUN Test_Validate_Calico/AWS_source/destination_checks_enabled === RUN Test_Validate_Calico/AWS_source/destination_checks_disabled === RUN Test_Validate_Calico/AWS_source/destination_checks_left_as_is === RUN Test_Validate_Calico/encapsulation_none_with_IPv4 === RUN Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 === RUN Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 === RUN Test_Validate_Calico/unknown_Calico_IPIP_mode === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPv6_without_encapsulation --- PASS: Test_Validate_Calico (0.00s) --- PASS: Test_Validate_Calico/empty_specs (0.00s) --- PASS: Test_Validate_Calico/positive_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/negative_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/with_etcd_version (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_off (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_enabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_disabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_left_as_is (0.00s) --- PASS: Test_Validate_Calico/encapsulation_none_with_IPv4 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/unknown_Calico_IPIP_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPv6_without_encapsulation (0.00s) === RUN Test_Validate_Cilium --- PASS: Test_Validate_Cilium (0.00s) === RUN Test_Validate_RollingUpdate --- PASS: Test_Validate_RollingUpdate (0.00s) === RUN Test_Validate_NodeLocalDNS --- PASS: Test_Validate_NodeLocalDNS (0.00s) === RUN Test_Validate_CloudConfiguration === RUN Test_Validate_CloudConfiguration/neither === RUN Test_Validate_CloudConfiguration/all_false === RUN Test_Validate_CloudConfiguration/all_true === RUN Test_Validate_CloudConfiguration/os_false === RUN Test_Validate_CloudConfiguration/os_true === RUN Test_Validate_CloudConfiguration/all_false,_os_false === RUN Test_Validate_CloudConfiguration/all_false,_os_true === RUN Test_Validate_CloudConfiguration/all_true,_os_false === RUN Test_Validate_CloudConfiguration/all_true,_os_true --- PASS: Test_Validate_CloudConfiguration (0.00s) --- PASS: Test_Validate_CloudConfiguration/neither (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_true (0.00s) === RUN TestValidateSAExternalPermissions === RUN TestValidateSAExternalPermissions/Duplicate_SA === RUN TestValidateSAExternalPermissions/Missing_permissions === RUN TestValidateSAExternalPermissions/Setting_both_arn_and_inline === RUN TestValidateSAExternalPermissions/Empty_SA_name === RUN TestValidateSAExternalPermissions/Empty_SA_namespace --- PASS: TestValidateSAExternalPermissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Duplicate_SA (0.00s) --- PASS: TestValidateSAExternalPermissions/Missing_permissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Setting_both_arn_and_inline (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_name (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_namespace (0.00s) === RUN Test_Validate_Nvidia_Cluster --- PASS: Test_Validate_Nvidia_Cluster (0.00s) === RUN Test_Validate_Nvidia_Ig --- PASS: Test_Validate_Nvidia_Ig (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/validation 0.422s ? k8s.io/kops/pkg/apis/nodeup [no test files] === RUN TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub W0811 05:31:28.502885 33367 builder.go:215] failed to digest image "proxy.example.com/weaveworks/weave-kube" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub (0.01s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub W0811 05:31:28.504225 33367 builder.go:215] failed to digest image "proxy.example.com/debian" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL W0811 05:31:28.505648 33367 builder.go:215] failed to digest image "proxy.example.com/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL W0811 05:31:28.507067 33367 builder.go:215] failed to digest image "proxy.example.com/google_containers/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags W0811 05:31:28.519184 33367 builder.go:215] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags (0.01s) === RUN TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges W0811 05:31:28.520368 33367 builder.go:215] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" W0811 05:31:28.522405 33367 builder.go:215] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges (0.00s) === RUN TestRemapEmptySection --- PASS: TestRemapEmptySection (0.00s) === RUN Test_BuildVFSPath E0811 05:31:28.523137 33367 copyfile.go:205] Unable to determine VFS path from supplied URL: https://foo/k8s-for-greeks-kops/kubernetes-release/release/v1.7.2/bin/linux/amd64/kubectl E0811 05:31:28.523148 33367 copyfile.go:206] S3, Google Cloud Storage, and File Paths are supported. E0811 05:31:28.523157 33367 copyfile.go:207] For S3, please make sure that the supplied file repository URL adhere to S3 naming conventions, https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region. E0811 05:31:28.523166 33367 copyfile.go:208] For GCS, please make sure that the supplied file repository URL adheres to https://storage.googleapis.com/ --- PASS: Test_BuildVFSPath (0.00s) PASS ok k8s.io/kops/pkg/assets 0.071s === RUN TestBackoff --- PASS: TestBackoff (0.00s) PASS ok k8s.io/kops/pkg/backoff 0.034s ? k8s.io/kops/pkg/bootstrap [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/simple [no test files] ? k8s.io/kops/pkg/client/simple/api [no test files] === RUN TestSSHCredentialStoreOnConfigBase --- PASS: TestSSHCredentialStoreOnConfigBase (0.00s) === RUN TestSSHCredentialStoreOnOwnCFS --- PASS: TestSSHCredentialStoreOnOwnCFS (0.00s) PASS ok k8s.io/kops/pkg/client/simple/vfsclientset 0.156s === RUN TestToAzureVMName === RUN TestToAzureVMName/test_case_0 === RUN TestToAzureVMName/test_case_1 --- PASS: TestToAzureVMName (0.00s) --- PASS: TestToAzureVMName/test_case_0 (0.00s) --- PASS: TestToAzureVMName/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/cloudinstances 0.048s ? k8s.io/kops/pkg/clouds [no test files] ? k8s.io/kops/pkg/clusteraddons [no test files] === RUN TestSetClusterBadInput --- PASS: TestSetClusterBadInput (0.00s) === RUN TestSetClusterFields --- PASS: TestSetClusterFields (0.00s) === RUN TestSetCiliumFields --- PASS: TestSetCiliumFields (0.00s) === RUN TestSetInstanceGroupsBadInput --- PASS: TestSetInstanceGroupsBadInput (0.00s) === RUN TestSetInstanceGroupsFields --- PASS: TestSetInstanceGroupsFields (0.00s) === RUN TestUnsetClusterBadInput --- PASS: TestUnsetClusterBadInput (0.00s) === RUN TestUnsetClusterFields --- PASS: TestUnsetClusterFields (0.00s) === RUN TestUnsetCiliumFields --- PASS: TestUnsetCiliumFields (0.00s) === RUN TestUnsetInstanceGroupsBadInput --- PASS: TestUnsetInstanceGroupsBadInput (0.00s) === RUN TestUnsetInstanceGroupsFields --- PASS: TestUnsetInstanceGroupsFields (0.00s) PASS ok k8s.io/kops/pkg/commands 0.070s ? k8s.io/kops/pkg/commands/commandutils [no test files] ? k8s.io/kops/pkg/commands/helpers [no test files] === RUN TestGetStructVal --- PASS: TestGetStructVal (0.00s) === RUN TestWrongStructField --- PASS: TestWrongStructField (0.00s) PASS ok k8s.io/kops/pkg/configbuilder 0.081s ? k8s.io/kops/pkg/configserver [no test files] === RUN Test_Diff_1 --- PASS: Test_Diff_1 (0.00s) === RUN Test_Diff_2 --- PASS: Test_Diff_2 (0.00s) === RUN Test_Diff_3 --- PASS: Test_Diff_3 (0.00s) === RUN Test_Diff_ChangedLine --- PASS: Test_Diff_ChangedLine (0.00s) === RUN Test_Diff_4 --- PASS: Test_Diff_4 (0.00s) PASS ok k8s.io/kops/pkg/diff 0.097s === RUN TestIsGossipHostname --- PASS: TestIsGossipHostname (0.00s) PASS ok k8s.io/kops/pkg/dns 0.057s ? k8s.io/kops/pkg/dump [no test files] === RUN TestHasExtraFields --- PASS: TestHasExtraFields (0.01s) PASS ok k8s.io/kops/pkg/edit 0.037s === RUN TestFlagToFalse I0811 05:30:57.129654 16183 featureflag_test.go:33] Created flag Unittest1 I0811 05:30:57.129716 16183 featureflag.go:162] FeatureFlag "UnitTest1"=false I0811 05:30:57.129738 16183 featureflag.go:162] FeatureFlag "UnitTest1"=true --- PASS: TestFlagToFalse (0.00s) === RUN TestSetenv I0811 05:30:57.129808 16183 featureflag.go:162] FeatureFlag "UnitTest2"=false --- PASS: TestSetenv (0.00s) PASS ok k8s.io/kops/pkg/featureflag 0.083s === RUN TestBuildKCMFlags --- PASS: TestBuildKCMFlags (0.00s) === RUN TestKubeletConfigSpec --- PASS: TestKubeletConfigSpec (0.00s) === RUN TestBuildAPIServerFlags --- PASS: TestBuildAPIServerFlags (0.00s) PASS ok k8s.io/kops/pkg/flagbuilder 0.033s === RUN TestRenderInstanceGroupZones W0811 05:31:28.596935 33491 instancegroup.go:42] error fetching zones for instancegroup: cannot find subnet "badsubnet" (declared in instance group "", not found in cluster) --- PASS: TestRenderInstanceGroupZones (0.00s) === RUN TestRenderInstanceGroupSubnets --- PASS: TestRenderInstanceGroupSubnets (0.00s) PASS ok k8s.io/kops/pkg/formatter 0.056s === RUN TestRollingUpdateDisabledSurgeOS rollingupdate_os_test.go:100: Failing in new release PR when build is not yet published --- SKIP: TestRollingUpdateDisabledSurgeOS (0.00s) === RUN TestRollingUpdateAllNeedUpdate I0811 05:32:10.829880 40662 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0811 05:32:10.829949 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:10.831155 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.831209 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:10.836450 40662 instancegroups.go:503] Cluster validated. I0811 05:32:10.836472 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.836483 40662 instancegroups.go:503] Cluster validated. I0811 05:32:10.836491 40662 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0811 05:32:10.837919 40662 instancegroups.go:400] Draining the node: "master-1a.local". I0811 05:32:10.838424 40662 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0811 05:32:10.838445 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:10.839626 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.839671 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:10.844924 40662 instancegroups.go:503] Cluster validated. I0811 05:32:10.844951 40662 instancegroups.go:400] Draining the node: "master-1b.local". I0811 05:32:10.845402 40662 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0811 05:32:10.845424 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:10.846586 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.846612 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:10.851854 40662 instancegroups.go:503] Cluster validated. I0811 05:32:10.851880 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.851904 40662 instancegroups.go:503] Cluster validated. I0811 05:32:10.851913 40662 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0811 05:32:10.852581 40662 instancegroups.go:400] Draining the node: "node-1a.local". I0811 05:32:10.853102 40662 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0811 05:32:10.853127 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:10.854305 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.854324 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:10.859548 40662 instancegroups.go:503] Cluster validated. I0811 05:32:10.859582 40662 instancegroups.go:400] Draining the node: "node-1b.local". I0811 05:32:10.860209 40662 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0811 05:32:10.860237 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:10.861416 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.861444 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:10.866687 40662 instancegroups.go:503] Cluster validated. I0811 05:32:10.866738 40662 instancegroups.go:400] Draining the node: "node-1c.local". I0811 05:32:10.867478 40662 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0811 05:32:10.867510 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:10.868699 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.868734 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:10.873992 40662 instancegroups.go:503] Cluster validated. I0811 05:32:10.874021 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.874039 40662 instancegroups.go:503] Cluster validated. I0811 05:32:10.874053 40662 instancegroups.go:311] Tainting 3 nodes in "node-2" instancegroup. I0811 05:32:10.875103 40662 instancegroups.go:400] Draining the node: "node-2a.local". I0811 05:32:10.875865 40662 instancegroups.go:591] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0811 05:32:10.875919 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:10.877120 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.877170 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:10.882457 40662 instancegroups.go:503] Cluster validated. I0811 05:32:10.882541 40662 instancegroups.go:400] Draining the node: "node-2b.local". I0811 05:32:10.883403 40662 instancegroups.go:591] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0811 05:32:10.883441 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:10.884611 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.884647 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:10.889892 40662 instancegroups.go:503] Cluster validated. I0811 05:32:10.889952 40662 instancegroups.go:400] Draining the node: "node-2c.local". I0811 05:32:10.890729 40662 instancegroups.go:591] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0811 05:32:10.890766 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:10.891988 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.892023 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:10.897257 40662 instancegroups.go:503] Cluster validated. I0811 05:32:10.897292 40662 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdate (0.07s) === RUN TestRollingUpdateAllNeedUpdateCloudonly I0811 05:32:10.897774 40662 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0811 05:32:10.897810 40662 instancegroups.go:436] waiting for 1ms after terminating instance W0811 05:32:10.898951 40662 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0811 05:32:10.899010 40662 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0811 05:32:10.899090 40662 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0811 05:32:10.899111 40662 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0811 05:32:10.899141 40662 instancegroups.go:436] waiting for 1ms after terminating instance W0811 05:32:10.900326 40662 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0811 05:32:10.900382 40662 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0811 05:32:10.900397 40662 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0811 05:32:10.900426 40662 instancegroups.go:436] waiting for 1ms after terminating instance W0811 05:32:10.901585 40662 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0811 05:32:10.901620 40662 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0811 05:32:10.901670 40662 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0811 05:32:10.901693 40662 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0811 05:32:10.901726 40662 instancegroups.go:436] waiting for 1ms after terminating instance W0811 05:32:10.902916 40662 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0811 05:32:10.902968 40662 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0811 05:32:10.902983 40662 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0811 05:32:10.903011 40662 instancegroups.go:436] waiting for 1ms after terminating instance W0811 05:32:10.904192 40662 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0811 05:32:10.904243 40662 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0811 05:32:10.904257 40662 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0811 05:32:10.904293 40662 instancegroups.go:436] waiting for 1ms after terminating instance W0811 05:32:10.905496 40662 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0811 05:32:10.905524 40662 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0811 05:32:10.905571 40662 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0811 05:32:10.905585 40662 instancegroups.go:591] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0811 05:32:10.905612 40662 instancegroups.go:436] waiting for 1ms after terminating instance W0811 05:32:10.906784 40662 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0811 05:32:10.906836 40662 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0811 05:32:10.906851 40662 instancegroups.go:591] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0811 05:32:10.906889 40662 instancegroups.go:436] waiting for 1ms after terminating instance W0811 05:32:10.908097 40662 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0811 05:32:10.908148 40662 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0811 05:32:10.908163 40662 instancegroups.go:591] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0811 05:32:10.908191 40662 instancegroups.go:436] waiting for 1ms after terminating instance W0811 05:32:10.909359 40662 instancegroups.go:468] Not validating cluster as cloudonly flag is set. I0811 05:32:10.909392 40662 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateCloudonly (0.01s) === RUN TestRollingUpdateAllNeedUpdateNoFailOnValidate I0811 05:32:10.909690 40662 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0811 05:32:10.909722 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:10.910890 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.910930 40662 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0811 05:32:10.910954 40662 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0811 05:32:10.911013 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.911038 40662 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0811 05:32:10.911053 40662 instancegroups.go:479] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0811 05:32:10.911068 40662 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0811 05:32:10.911811 40662 instancegroups.go:400] Draining the node: "master-1a.local". I0811 05:32:10.912631 40662 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0811 05:32:10.912676 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:10.913890 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.913932 40662 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0811 05:32:10.913953 40662 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0811 05:32:10.914005 40662 instancegroups.go:400] Draining the node: "master-1b.local". I0811 05:32:10.914841 40662 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0811 05:32:10.914882 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:10.916068 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.916111 40662 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0811 05:32:10.916133 40662 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0811 05:32:10.916154 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.916174 40662 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0811 05:32:10.916188 40662 instancegroups.go:479] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0811 05:32:10.916202 40662 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0811 05:32:10.917188 40662 instancegroups.go:400] Draining the node: "node-1a.local". I0811 05:32:10.917967 40662 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0811 05:32:10.918007 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:10.919195 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.919242 40662 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0811 05:32:10.919264 40662 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0811 05:32:10.919317 40662 instancegroups.go:400] Draining the node: "node-1b.local". I0811 05:32:10.920196 40662 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0811 05:32:10.920235 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:10.921395 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.921439 40662 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0811 05:32:10.921461 40662 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0811 05:32:10.921505 40662 instancegroups.go:400] Draining the node: "node-1c.local". I0811 05:32:10.922454 40662 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0811 05:32:10.922496 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:10.923601 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.923643 40662 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0811 05:32:10.923664 40662 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0811 05:32:10.923679 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.923707 40662 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0811 05:32:10.923730 40662 instancegroups.go:479] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0811 05:32:10.923744 40662 instancegroups.go:311] Tainting 3 nodes in "node-2" instancegroup. I0811 05:32:10.924918 40662 instancegroups.go:400] Draining the node: "node-2a.local". I0811 05:32:10.925759 40662 instancegroups.go:591] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0811 05:32:10.925798 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:10.926905 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.926947 40662 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0811 05:32:10.926969 40662 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0811 05:32:10.927008 40662 instancegroups.go:400] Draining the node: "node-2b.local". I0811 05:32:10.927826 40662 instancegroups.go:591] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0811 05:32:10.927865 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:10.928999 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.929063 40662 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0811 05:32:10.929097 40662 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0811 05:32:10.929147 40662 instancegroups.go:400] Draining the node: "node-2c.local". I0811 05:32:10.929940 40662 instancegroups.go:591] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0811 05:32:10.929988 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:10.931197 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.931242 40662 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0811 05:32:10.931267 40662 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0811 05:32:10.931286 40662 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateNoFailOnValidate (0.02s) === RUN TestRollingUpdateNoneNeedUpdate I0811 05:32:10.931614 40662 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdate (0.00s) === RUN TestRollingUpdateNoneNeedUpdateWithForce I0811 05:32:10.931883 40662 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0811 05:32:10.931916 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:10.933055 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.933090 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:10.938347 40662 instancegroups.go:503] Cluster validated. I0811 05:32:10.938409 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.938431 40662 instancegroups.go:503] Cluster validated. I0811 05:32:10.938445 40662 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0811 05:32:10.939227 40662 instancegroups.go:400] Draining the node: "master-1a.local". I0811 05:32:10.940020 40662 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0811 05:32:10.940060 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:10.941268 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.941311 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:10.946606 40662 instancegroups.go:503] Cluster validated. I0811 05:32:10.946656 40662 instancegroups.go:400] Draining the node: "master-1b.local". I0811 05:32:10.947502 40662 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0811 05:32:10.947545 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:10.948763 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.948805 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:10.954072 40662 instancegroups.go:503] Cluster validated. I0811 05:32:10.954120 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.954140 40662 instancegroups.go:503] Cluster validated. I0811 05:32:10.954153 40662 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0811 05:32:10.955166 40662 instancegroups.go:400] Draining the node: "node-1a.local". I0811 05:32:10.955928 40662 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0811 05:32:10.955968 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:10.957184 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.957220 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:10.962481 40662 instancegroups.go:503] Cluster validated. I0811 05:32:10.962544 40662 instancegroups.go:400] Draining the node: "node-1b.local". I0811 05:32:10.963391 40662 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0811 05:32:10.963433 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:10.964644 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.964679 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:10.969949 40662 instancegroups.go:503] Cluster validated. I0811 05:32:10.970005 40662 instancegroups.go:400] Draining the node: "node-1c.local". I0811 05:32:10.970808 40662 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0811 05:32:10.970848 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:10.972056 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.972091 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:10.978310 40662 instancegroups.go:503] Cluster validated. I0811 05:32:10.978368 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.978395 40662 instancegroups.go:503] Cluster validated. I0811 05:32:10.978414 40662 instancegroups.go:311] Tainting 3 nodes in "node-2" instancegroup. I0811 05:32:10.979654 40662 instancegroups.go:400] Draining the node: "node-2a.local". I0811 05:32:10.980489 40662 instancegroups.go:591] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0811 05:32:10.980534 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:10.981748 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.981787 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:10.987034 40662 instancegroups.go:503] Cluster validated. I0811 05:32:10.987086 40662 instancegroups.go:400] Draining the node: "node-2b.local". I0811 05:32:10.988031 40662 instancegroups.go:591] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0811 05:32:10.988068 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:10.989259 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.989296 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:10.994543 40662 instancegroups.go:503] Cluster validated. I0811 05:32:10.994594 40662 instancegroups.go:400] Draining the node: "node-2c.local". I0811 05:32:10.995630 40662 instancegroups.go:591] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0811 05:32:10.995666 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:10.996884 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:10.996921 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.002171 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.002207 40662 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdateWithForce (0.07s) === RUN TestRollingUpdateEmptyGroup I0811 05:32:11.002443 40662 rollingupdate.go:96] Cloud Instance Group length is zero. Not doing a rolling-update. --- PASS: TestRollingUpdateEmptyGroup (0.00s) === RUN TestRollingUpdateUnknownRole --- PASS: TestRollingUpdateUnknownRole (0.00s) === RUN TestRollingUpdateAllNeedUpdateFailsValidation I0811 05:32:11.003036 40662 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0811 05:32:11.003068 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.004271 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.004314 40662 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. E0811 05:32:11.004335 40662 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateFailsValidation (0.00s) === RUN TestRollingUpdateAllNeedUpdateErrorsValidation I0811 05:32:11.004754 40662 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0811 05:32:11.004788 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.005977 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.006014 40662 instancegroups.go:513] Cluster did not validate within deadline: testing validation error. E0811 05:32:11.006065 40662 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateFailsValidation I0811 05:32:11.006387 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.006418 40662 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. E0811 05:32:11.006435 40662 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateNodes1NeedsUpdateFailsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateErrorsValidation I0811 05:32:11.006626 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.006652 40662 instancegroups.go:513] Cluster did not validate within deadline: testing validation error. E0811 05:32:11.006668 40662 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateNodes1NeedsUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneMaster I0811 05:32:11.006940 40662 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0811 05:32:11.006971 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.008196 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.008242 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.013526 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.013588 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.013612 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.013626 40662 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0811 05:32:11.014481 40662 instancegroups.go:400] Draining the node: "master-1a.local". I0811 05:32:11.015392 40662 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0811 05:32:11.015429 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.016635 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.016684 40662 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. E0811 05:32:11.016706 40662 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneMaster I0811 05:32:11.017076 40662 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0811 05:32:11.017118 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.018328 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.018401 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.023721 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.023785 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.023810 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.023823 40662 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0811 05:32:11.024766 40662 instancegroups.go:400] Draining the node: "master-1a.local". I0811 05:32:11.025672 40662 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0811 05:32:11.025728 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.026929 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.026969 40662 instancegroups.go:513] Cluster did not validate within deadline: testing validation error. E0811 05:32:11.026990 40662 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateNonRelatedInstanceGroupFailure I0811 05:32:11.027349 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.027388 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.027403 40662 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0811 05:32:11.028531 40662 instancegroups.go:400] Draining the node: "node-1a.local". I0811 05:32:11.029401 40662 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0811 05:32:11.029445 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.030651 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.030689 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.036097 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.036146 40662 instancegroups.go:400] Draining the node: "node-1b.local". I0811 05:32:11.037146 40662 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0811 05:32:11.037185 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.038391 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.038429 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.043670 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.043738 40662 instancegroups.go:400] Draining the node: "node-1c.local". I0811 05:32:11.044787 40662 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0811 05:32:11.044823 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.046403 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.046442 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.051741 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.051777 40662 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNonRelatedInstanceGroupFailure (0.02s) === RUN TestRollingUpdateRelatedInstanceGroupFailure I0811 05:32:11.052153 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.052184 40662 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. E0811 05:32:11.052203 40662 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateRelatedInstanceGroupFailure (0.00s) === RUN TestRollingUpdateMasterGroupFailure I0811 05:32:11.052519 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.052548 40662 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. E0811 05:32:11.052565 40662 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateMasterGroupFailure (0.00s) === RUN TestRollingUpdateValidationErrorInstanceGroupNil I0811 05:32:11.052847 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.052875 40662 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. E0811 05:32:11.052891 40662 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateValidationErrorInstanceGroupNil (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneNode I0811 05:32:11.053125 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.053154 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.053167 40662 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0811 05:32:11.054419 40662 instancegroups.go:400] Draining the node: "node-1a.local". I0811 05:32:11.055334 40662 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0811 05:32:11.055369 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.056575 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.056624 40662 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. E0811 05:32:11.056646 40662 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneNode I0811 05:32:11.056927 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.056960 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.056973 40662 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0811 05:32:11.058226 40662 instancegroups.go:400] Draining the node: "node-1a.local". I0811 05:32:11.059140 40662 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0811 05:32:11.059179 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.060390 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.060434 40662 instancegroups.go:513] Cluster did not validate within deadline: testing validation error. E0811 05:32:11.060457 40662 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateFlappingValidation I0811 05:32:11.060803 40662 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0811 05:32:11.060834 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.062054 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.062158 40662 instancegroups.go:526] Cluster did not pass validation, will retry in "1ms": testing failure. I0811 05:32:11.063406 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.068695 40662 instancegroups.go:526] Cluster did not pass validation, will retry in "1ms": testing failure. I0811 05:32:11.069920 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.075160 40662 instancegroups.go:526] Cluster did not pass validation, will retry in "1ms": testing failure. I0811 05:32:11.076336 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.081579 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.081647 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.081679 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.081693 40662 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0811 05:32:11.082608 40662 instancegroups.go:400] Draining the node: "master-1a.local". I0811 05:32:11.083513 40662 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0811 05:32:11.083554 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.084765 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.084808 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.090077 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.090133 40662 instancegroups.go:400] Draining the node: "master-1b.local". I0811 05:32:11.092067 40662 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0811 05:32:11.092121 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.093322 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.093391 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.098702 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.098764 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.098819 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.098845 40662 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0811 05:32:11.100528 40662 instancegroups.go:400] Draining the node: "node-1a.local". I0811 05:32:11.101378 40662 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0811 05:32:11.101421 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.102599 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.102649 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.107913 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.107972 40662 instancegroups.go:400] Draining the node: "node-1b.local". I0811 05:32:11.108815 40662 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0811 05:32:11.108856 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.110067 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.110126 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.115389 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.115449 40662 instancegroups.go:400] Draining the node: "node-1c.local". I0811 05:32:11.116395 40662 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0811 05:32:11.116438 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.117631 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.117679 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.122949 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.122982 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.123010 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.123024 40662 instancegroups.go:311] Tainting 3 nodes in "node-2" instancegroup. I0811 05:32:11.124168 40662 instancegroups.go:400] Draining the node: "node-2a.local". I0811 05:32:11.125040 40662 instancegroups.go:591] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0811 05:32:11.125080 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.126270 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.126319 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.131580 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.131641 40662 instancegroups.go:400] Draining the node: "node-2b.local". I0811 05:32:11.132606 40662 instancegroups.go:591] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0811 05:32:11.132646 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.133848 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.133901 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.139194 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.139268 40662 instancegroups.go:400] Draining the node: "node-2c.local". I0811 05:32:11.140167 40662 instancegroups.go:591] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0811 05:32:11.140208 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.141403 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.141451 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.147573 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.147627 40662 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateFlappingValidation (0.09s) === RUN TestRollingUpdateValidatesAfterBastion I0811 05:32:11.148075 40662 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0811 05:32:11.148111 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.149277 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.149318 40662 instancegroups.go:526] Cluster did not pass validation, will retry in "1ms": testing failure. I0811 05:32:11.150494 40662 instancegroups.go:526] Cluster did not pass validation, will retry in "1ms": testing failure. I0811 05:32:11.151699 40662 instancegroups.go:526] Cluster did not pass validation, will retry in "1ms": testing failure. I0811 05:32:11.152834 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.158101 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.158196 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.158238 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.158271 40662 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0811 05:32:11.159628 40662 instancegroups.go:400] Draining the node: "master-1a.local". I0811 05:32:11.161216 40662 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0811 05:32:11.161281 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.162396 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.162439 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.167673 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.167732 40662 instancegroups.go:400] Draining the node: "master-1b.local". I0811 05:32:11.168581 40662 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0811 05:32:11.168624 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.169828 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.169888 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.175141 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.175182 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.175203 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.175218 40662 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0811 05:32:11.176399 40662 instancegroups.go:400] Draining the node: "node-1a.local". I0811 05:32:11.177285 40662 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0811 05:32:11.177326 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.178503 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.178538 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.183776 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.183834 40662 instancegroups.go:400] Draining the node: "node-1b.local". I0811 05:32:11.184758 40662 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0811 05:32:11.184799 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.185989 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.186025 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.191286 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.191344 40662 instancegroups.go:400] Draining the node: "node-1c.local". I0811 05:32:11.192197 40662 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0811 05:32:11.192238 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.193438 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.193474 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.198735 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.198782 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.198817 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.198840 40662 instancegroups.go:311] Tainting 3 nodes in "node-2" instancegroup. I0811 05:32:11.200445 40662 instancegroups.go:400] Draining the node: "node-2a.local". I0811 05:32:11.201670 40662 instancegroups.go:591] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0811 05:32:11.201714 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.202905 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.202953 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.208219 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.208281 40662 instancegroups.go:400] Draining the node: "node-2b.local". I0811 05:32:11.209234 40662 instancegroups.go:591] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0811 05:32:11.209276 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.210487 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.210522 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.215765 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.215824 40662 instancegroups.go:400] Draining the node: "node-2c.local". I0811 05:32:11.216696 40662 instancegroups.go:591] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0811 05:32:11.216736 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.217949 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.217985 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.223256 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.223292 40662 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateValidatesAfterBastion (0.08s) === RUN TestAddAnnotatedNodesToNeedsUpdate --- PASS: TestAddAnnotatedNodesToNeedsUpdate (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateCloudonly --- PASS: TestAddAnnotatedNodesToNeedsUpdateCloudonly (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateNodesMissing --- PASS: TestAddAnnotatedNodesToNeedsUpdateNodesMissing (0.00s) === RUN TestRollingUpdateTaintAllButOneNeedUpdate I0811 05:32:11.224028 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.224056 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.224069 40662 instancegroups.go:311] Tainting 2 nodes in "node-1" instancegroup. I0811 05:32:11.224842 40662 instancegroups.go:400] Draining the node: "node-1a.local". I0811 05:32:11.226007 40662 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0811 05:32:11.226061 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.227236 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.227296 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.232744 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.232803 40662 instancegroups.go:400] Draining the node: "node-1b.local". I0811 05:32:11.233654 40662 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0811 05:32:11.233694 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.234866 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.234933 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.240224 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.240258 40662 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateTaintAllButOneNeedUpdate (0.02s) === RUN TestRollingUpdateMaxSurgeIgnoredForMaster I0811 05:32:11.240544 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.240573 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.240587 40662 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0811 05:32:11.241324 40662 instancegroups.go:400] Draining the node: "master-1a.local". I0811 05:32:11.242080 40662 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0811 05:32:11.242114 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.243306 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.243353 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.248647 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.248702 40662 instancegroups.go:400] Draining the node: "master-1b.local". I0811 05:32:11.249560 40662 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0811 05:32:11.249600 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.250770 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.250816 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.256795 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.256853 40662 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeIgnoredForMaster (0.02s) === RUN TestRollingUpdateDisabled I0811 05:32:11.257327 40662 instancegroups.go:185] Rolling updates for InstanceGroup bastion-1 are disabled W0811 05:32:11.257378 40662 instancegroups.go:468] Not validating cluster as cloudonly flag is set. I0811 05:32:11.257399 40662 instancegroups.go:185] Rolling updates for InstanceGroup master-1 are disabled W0811 05:32:11.257416 40662 instancegroups.go:468] Not validating cluster as cloudonly flag is set. I0811 05:32:11.257435 40662 instancegroups.go:185] Rolling updates for InstanceGroup node-1 are disabled W0811 05:32:11.257451 40662 instancegroups.go:468] Not validating cluster as cloudonly flag is set. I0811 05:32:11.257472 40662 instancegroups.go:185] Rolling updates for InstanceGroup node-2 are disabled I0811 05:32:11.257488 40662 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabled (0.00s) === RUN TestRollingUpdateDisabledSurge I0811 05:32:11.257776 40662 instancegroups.go:570] Detaching instance "bastion-1a", in group "bastion-1". I0811 05:32:11.257837 40662 instancegroups.go:172] waiting for 1ms after detaching instance I0811 05:32:11.258975 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.259009 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.264177 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.264214 40662 instancegroups.go:185] Rolling updates for InstanceGroup bastion-1 are disabled I0811 05:32:11.264265 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.264290 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.264308 40662 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0811 05:32:11.265166 40662 instancegroups.go:185] Rolling updates for InstanceGroup master-1 are disabled I0811 05:32:11.265194 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.265214 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.265226 40662 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0811 05:32:11.266169 40662 instancegroups.go:568] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0811 05:32:11.266227 40662 instancegroups.go:172] waiting for 1ms after detaching instance I0811 05:32:11.267431 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.267467 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.272681 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.272715 40662 instancegroups.go:185] Rolling updates for InstanceGroup node-1 are disabled I0811 05:32:11.272734 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.272755 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.272773 40662 instancegroups.go:311] Tainting 3 nodes in "node-2" instancegroup. I0811 05:32:11.273738 40662 instancegroups.go:568] Detaching instance "node-2c", node "node-2c.local", in group "node-2". I0811 05:32:11.273806 40662 instancegroups.go:172] waiting for 1ms after detaching instance I0811 05:32:11.275040 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.275075 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:11.280313 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.280351 40662 instancegroups.go:185] Rolling updates for InstanceGroup node-2 are disabled I0811 05:32:11.280369 40662 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabledSurge (0.02s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdate I0811 05:32:11.280621 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.280659 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.280674 40662 instancegroups.go:311] Tainting 7 nodes in "node-1" instancegroup. I0811 05:32:11.282838 40662 instancegroups.go:400] Draining the node: "node-1a.local". I0811 05:32:11.283593 40662 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0811 05:32:11.283642 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.284884 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.284930 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.284978 40662 instancegroups.go:400] Draining the node: "node-1c.local". I0811 05:32:11.285044 40662 instancegroups.go:400] Draining the node: "node-1b.local". I0811 05:32:11.286436 40662 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0811 05:32:11.286730 40662 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0811 05:32:11.286776 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.288005 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.288084 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.388595 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.388682 40662 instancegroups.go:400] Draining the node: "node-1e.local". I0811 05:32:11.388726 40662 instancegroups.go:400] Draining the node: "node-1d.local". I0811 05:32:11.390035 40662 instancegroups.go:591] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0811 05:32:11.390324 40662 instancegroups.go:591] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0811 05:32:11.390367 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.391577 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.391621 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.391660 40662 instancegroups.go:400] Draining the node: "node-1f.local". I0811 05:32:11.392556 40662 instancegroups.go:591] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0811 05:32:11.392600 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.393816 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.393876 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.494362 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.494417 40662 instancegroups.go:400] Draining the node: "node-1g.local". I0811 05:32:11.495259 40662 instancegroups.go:591] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0811 05:32:11.495304 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.496520 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.496563 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.496580 40662 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxUnavailableAllButOneNeedUpdate I0811 05:32:11.496819 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.496853 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.496867 40662 instancegroups.go:311] Tainting 6 nodes in "node-1" instancegroup. I0811 05:32:11.498677 40662 instancegroups.go:400] Draining the node: "node-1b.local". I0811 05:32:11.498759 40662 instancegroups.go:400] Draining the node: "node-1a.local". I0811 05:32:11.500137 40662 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0811 05:32:11.500423 40662 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0811 05:32:11.500466 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.501682 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.501749 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.602223 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.602281 40662 instancegroups.go:400] Draining the node: "node-1d.local". I0811 05:32:11.602315 40662 instancegroups.go:400] Draining the node: "node-1c.local". I0811 05:32:11.602965 40662 instancegroups.go:591] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0811 05:32:11.603415 40662 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0811 05:32:11.603446 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.604655 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.604712 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.604761 40662 instancegroups.go:400] Draining the node: "node-1e.local". I0811 05:32:11.605624 40662 instancegroups.go:591] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0811 05:32:11.605678 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.606881 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.606922 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.707404 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.707476 40662 instancegroups.go:400] Draining the node: "node-1f.local". I0811 05:32:11.708154 40662 instancegroups.go:591] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0811 05:32:11.708193 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.709388 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.709439 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.709458 40662 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllButOneNeedUpdate (0.21s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdateMaster I0811 05:32:11.709720 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.709754 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.709769 40662 instancegroups.go:311] Tainting 7 nodes in "master-1" instancegroup. I0811 05:32:11.711861 40662 instancegroups.go:400] Draining the node: "master-1a.local". I0811 05:32:11.712644 40662 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0811 05:32:11.712695 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.713861 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.713889 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.713915 40662 instancegroups.go:400] Draining the node: "master-1c.local". I0811 05:32:11.713946 40662 instancegroups.go:400] Draining the node: "master-1b.local". I0811 05:32:11.714738 40662 instancegroups.go:591] Stopping instance "master-1c", node "master-1c.local", in group "master-1" (this may take a while). I0811 05:32:11.714904 40662 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0811 05:32:11.714931 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.716121 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.716196 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.816788 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.816865 40662 instancegroups.go:400] Draining the node: "master-1e.local". I0811 05:32:11.816966 40662 instancegroups.go:400] Draining the node: "master-1d.local". I0811 05:32:11.818361 40662 instancegroups.go:591] Stopping instance "master-1e", node "master-1e.local", in group "master-1" (this may take a while). I0811 05:32:11.818656 40662 instancegroups.go:591] Stopping instance "master-1d", node "master-1d.local", in group "master-1" (this may take a while). I0811 05:32:11.818706 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.819920 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.819972 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.820017 40662 instancegroups.go:400] Draining the node: "master-1f.local". I0811 05:32:11.821951 40662 instancegroups.go:591] Stopping instance "master-1f", node "master-1f.local", in group "master-1" (this may take a while). I0811 05:32:11.822020 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.823184 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.823259 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.923496 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.923564 40662 instancegroups.go:400] Draining the node: "master-1g.local". I0811 05:32:11.924668 40662 instancegroups.go:591] Stopping instance "master-1g", node "master-1g.local", in group "master-1" (this may take a while). I0811 05:32:11.924720 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.925951 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.926023 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.926045 40662 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdateMaster (0.22s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdate I0811 05:32:11.926383 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.926423 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.926438 40662 instancegroups.go:311] Tainting 6 nodes in "node-1" instancegroup. I0811 05:32:11.928275 40662 instancegroups.go:568] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0811 05:32:11.928345 40662 instancegroups.go:172] waiting for 1ms after detaching instance I0811 05:32:11.929525 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.929575 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.929590 40662 instancegroups.go:568] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0811 05:32:11.929654 40662 instancegroups.go:172] waiting for 1ms after detaching instance I0811 05:32:11.930871 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.930915 40662 instancegroups.go:503] Cluster validated. I0811 05:32:11.930963 40662 instancegroups.go:400] Draining the node: "node-1b.local". I0811 05:32:11.931026 40662 instancegroups.go:400] Draining the node: "node-1a.local". I0811 05:32:11.932392 40662 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0811 05:32:11.932821 40662 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0811 05:32:11.932900 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:11.934108 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:11.934178 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.034737 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.034836 40662 instancegroups.go:400] Draining the node: "node-1d.local". I0811 05:32:12.034893 40662 instancegroups.go:400] Draining the node: "node-1c.local". I0811 05:32:12.036361 40662 instancegroups.go:591] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0811 05:32:12.036645 40662 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0811 05:32:12.036691 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.037907 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.037951 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.038000 40662 instancegroups.go:400] Draining the node: "node-1e.local". I0811 05:32:12.038830 40662 instancegroups.go:591] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0811 05:32:12.038880 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.040088 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.040154 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.140783 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.140837 40662 instancegroups.go:400] Draining the node: "node-1f.local". I0811 05:32:12.141684 40662 instancegroups.go:591] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0811 05:32:12.141737 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.142959 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.143039 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.143062 40662 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeAllButOneNeedUpdate I0811 05:32:12.143383 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.143423 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.143439 40662 instancegroups.go:311] Tainting 6 nodes in "node-1" instancegroup. I0811 05:32:12.145298 40662 instancegroups.go:568] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0811 05:32:12.145353 40662 instancegroups.go:568] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0811 05:32:12.145400 40662 instancegroups.go:172] waiting for 1ms after detaching instance I0811 05:32:12.146603 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.146645 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.146682 40662 instancegroups.go:400] Draining the node: "node-1b.local". I0811 05:32:12.146708 40662 instancegroups.go:400] Draining the node: "node-1a.local". I0811 05:32:12.147472 40662 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0811 05:32:12.148051 40662 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0811 05:32:12.148093 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.149292 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.149374 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.249982 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.250033 40662 instancegroups.go:400] Draining the node: "node-1d.local". I0811 05:32:12.250044 40662 instancegroups.go:400] Draining the node: "node-1c.local". I0811 05:32:12.250857 40662 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0811 05:32:12.251540 40662 instancegroups.go:591] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0811 05:32:12.251589 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.252751 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.252801 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.252847 40662 instancegroups.go:400] Draining the node: "node-1e.local". I0811 05:32:12.253680 40662 instancegroups.go:591] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0811 05:32:12.253731 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.254903 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.254978 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.355678 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.355878 40662 instancegroups.go:400] Draining the node: "node-1f.local". I0811 05:32:12.356805 40662 instancegroups.go:591] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0811 05:32:12.356862 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.358076 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.358173 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.358210 40662 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllButOneNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeGreaterThanNeedUpdate I0811 05:32:12.358664 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.358711 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.358736 40662 instancegroups.go:311] Tainting 2 nodes in "node-1" instancegroup. I0811 05:32:12.360129 40662 instancegroups.go:568] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0811 05:32:12.360189 40662 instancegroups.go:568] Detaching instance "node-1a", node "node-1a.local", in group "node-1". I0811 05:32:12.360258 40662 instancegroups.go:172] waiting for 1ms after detaching instance I0811 05:32:12.361518 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.361556 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:12.366828 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.366908 40662 instancegroups.go:400] Draining the node: "node-1b.local". I0811 05:32:12.366914 40662 instancegroups.go:400] Draining the node: "node-1a.local". I0811 05:32:12.368379 40662 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0811 05:32:12.368418 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.368871 40662 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0811 05:32:12.368934 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.369621 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.369656 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:12.374940 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.374983 40662 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeGreaterThanNeedUpdate (0.02s) === RUN TestRollingUpdateDetachFails I0811 05:32:12.375308 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.375355 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.375384 40662 instancegroups.go:311] Tainting 2 nodes in "node-1" instancegroup. I0811 05:32:12.376753 40662 instancegroups.go:568] Detaching instance "node-1b", node "node-1b.local", in group "node-1". E0811 05:32:12.376854 40662 instancegroups.go:160] Failed to detach instance "node-1b": error detaching instance "node-1b", node "node-1b.local": error detaching instance "node-1b": testing error I0811 05:32:12.376883 40662 instancegroups.go:568] Detaching instance "node-1a", node "node-1a.local", in group "node-1". E0811 05:32:12.376955 40662 instancegroups.go:160] Failed to detach instance "node-1a": error detaching instance "node-1a", node "node-1a.local": error detaching instance "node-1a": testing error I0811 05:32:12.377016 40662 instancegroups.go:172] waiting for 1ms after detaching instance I0811 05:32:12.378251 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.378287 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:12.383543 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.383612 40662 instancegroups.go:400] Draining the node: "node-1b.local". I0811 05:32:12.383637 40662 instancegroups.go:400] Draining the node: "node-1a.local". I0811 05:32:12.385054 40662 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0811 05:32:12.385093 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.385494 40662 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0811 05:32:12.385551 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.386303 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.386339 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:12.391849 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.391886 40662 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDetachFails (0.02s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached I0811 05:32:12.392116 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.392155 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.392169 40662 instancegroups.go:311] Tainting 4 nodes in "node-1" instancegroup. I0811 05:32:12.393469 40662 instancegroups.go:568] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0811 05:32:12.393541 40662 instancegroups.go:172] waiting for 1ms after detaching instance I0811 05:32:12.395195 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.395241 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.395256 40662 instancegroups.go:568] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0811 05:32:12.395316 40662 instancegroups.go:172] waiting for 1ms after detaching instance I0811 05:32:12.396485 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.396531 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.396582 40662 instancegroups.go:400] Draining the node: "node-1c.local". I0811 05:32:12.396653 40662 instancegroups.go:400] Draining the node: "node-1a.local". I0811 05:32:12.396677 40662 instancegroups.go:400] Draining the node: "node-1b.local". I0811 05:32:12.398324 40662 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0811 05:32:12.398379 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.398661 40662 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0811 05:32:12.398716 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.398974 40662 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0811 05:32:12.399053 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.399575 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.500119 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.500195 40662 instancegroups.go:400] Draining the node: "node-1d.local". I0811 05:32:12.501142 40662 instancegroups.go:591] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0811 05:32:12.501195 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.502383 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.502446 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.502465 40662 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached (0.11s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached I0811 05:32:12.502719 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.502754 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.502771 40662 instancegroups.go:311] Tainting 7 nodes in "node-1" instancegroup. I0811 05:32:12.504878 40662 instancegroups.go:400] Draining the node: "node-1a.local". I0811 05:32:12.505660 40662 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0811 05:32:12.505727 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.506945 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.506999 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.507049 40662 instancegroups.go:400] Draining the node: "node-1e.local". I0811 05:32:12.507112 40662 instancegroups.go:400] Draining the node: "node-1c.local". I0811 05:32:12.508386 40662 instancegroups.go:591] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0811 05:32:12.508707 40662 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0811 05:32:12.508754 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.509971 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.510042 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.610889 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.610967 40662 instancegroups.go:400] Draining the node: "node-1g.local". I0811 05:32:12.610990 40662 instancegroups.go:400] Draining the node: "node-1f.local". I0811 05:32:12.612551 40662 instancegroups.go:591] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0811 05:32:12.612961 40662 instancegroups.go:591] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0811 05:32:12.613029 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.614281 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.614327 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.614372 40662 instancegroups.go:400] Draining the node: "node-1b.local". I0811 05:32:12.615281 40662 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0811 05:32:12.615328 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.616485 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.616555 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.717237 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.717328 40662 instancegroups.go:400] Draining the node: "node-1d.local". I0811 05:32:12.718309 40662 instancegroups.go:591] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0811 05:32:12.718369 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.719585 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.719650 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.719672 40662 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached (0.22s) === RUN TestRollingUpdateOnlyWarmPoolNodes I0811 05:32:12.719991 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.720019 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.720041 40662 instancegroups.go:132] deleting warm pool instance "node-1-wp-a" I0811 05:32:12.720079 40662 instancegroups.go:132] deleting warm pool instance "node-1-wp-b" I0811 05:32:12.720107 40662 instancegroups.go:132] deleting warm pool instance "node-1-wp-c" I0811 05:32:12.720134 40662 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateOnlyWarmPoolNodes (0.00s) === RUN TestRollingWarmPoolBeforeJoinedNodes I0811 05:32:12.720356 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.720383 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.720397 40662 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0811 05:32:12.721442 40662 instancegroups.go:132] deleting warm pool instance "node-1-wp-a" I0811 05:32:12.721476 40662 instancegroups.go:132] deleting warm pool instance "node-1-wp-b" I0811 05:32:12.721498 40662 instancegroups.go:132] deleting warm pool instance "node-1-wp-c" I0811 05:32:12.721544 40662 instancegroups.go:400] Draining the node: "node-1a.local". I0811 05:32:12.722370 40662 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0811 05:32:12.722421 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.723641 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.723682 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:12.728944 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.728995 40662 instancegroups.go:400] Draining the node: "node-1b.local". I0811 05:32:12.729849 40662 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0811 05:32:12.729887 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.731091 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.731127 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:12.736362 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.736413 40662 instancegroups.go:400] Draining the node: "node-1c.local". I0811 05:32:12.737244 40662 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0811 05:32:12.737280 40662 instancegroups.go:436] waiting for 1ms after terminating instance I0811 05:32:12.738498 40662 instancegroups.go:470] Validating the cluster. I0811 05:32:12.738534 40662 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0811 05:32:12.743775 40662 instancegroups.go:503] Cluster validated. I0811 05:32:12.743813 40662 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingWarmPoolBeforeJoinedNodes (0.02s) === RUN TestSettings === RUN TestSettings/DrainAndTerminate === RUN TestSettings/MaxUnavailable === RUN TestSettings/MaxSurge --- PASS: TestSettings (0.00s) --- PASS: TestSettings/DrainAndTerminate (0.00s) --- PASS: TestSettings/MaxUnavailable (0.00s) --- PASS: TestSettings/MaxSurge (0.00s) === RUN TestMaxUnavailable === RUN TestMaxUnavailable/0_1 === RUN TestMaxUnavailable/0%_1 === RUN TestMaxUnavailable/39%_10 === RUN TestMaxUnavailable/100%_10 --- PASS: TestMaxUnavailable (0.00s) --- PASS: TestMaxUnavailable/0_1 (0.00s) --- PASS: TestMaxUnavailable/0%_1 (0.00s) --- PASS: TestMaxUnavailable/39%_10 (0.00s) --- PASS: TestMaxUnavailable/100%_10 (0.00s) === RUN TestMaxSurge === RUN TestMaxSurge/0_1 === RUN TestMaxSurge/0%_1 === RUN TestMaxSurge/31%_10 === RUN TestMaxSurge/100%_10 --- PASS: TestMaxSurge (0.00s) --- PASS: TestMaxSurge/0_1 (0.00s) --- PASS: TestMaxSurge/0%_1 (0.00s) --- PASS: TestMaxSurge/31%_10 (0.00s) --- PASS: TestMaxSurge/100%_10 (0.00s) === RUN TestAWSDefault --- PASS: TestAWSDefault (0.00s) PASS ok k8s.io/kops/pkg/instancegroups 1.995s === RUN TestWriteToken === RUN TestWriteToken/WriteJson === RUN TestWriteToken/WriteJsonWithIndent --- PASS: TestWriteToken (0.00s) --- PASS: TestWriteToken/WriteJson (0.00s) --- PASS: TestWriteToken/WriteJsonWithIndent (0.00s) PASS ok k8s.io/kops/pkg/jsonutils 0.071s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.00s) PASS ok k8s.io/kops/pkg/k8scodecs 0.010s === RUN TestParse E0811 05:31:16.271989 26757 versions.go:96] unable to parse Kubernetes version "" E0811 05:31:16.272041 26757 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: TestParse (0.00s) === RUN TestIsGTE === RUN TestIsGTE/KV_greater_than_Version === RUN TestIsGTE/KV_greater_than_Version#01 === RUN TestIsGTE/KV_equal_Version === RUN TestIsGTE/KV_equal_Version#01 === RUN TestIsGTE/Version_greater_than_KV === RUN TestIsGTE/Version_greater_than_KV#01 --- PASS: TestIsGTE (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version#01 (0.00s) --- PASS: TestIsGTE/KV_equal_Version (0.00s) --- PASS: TestIsGTE/KV_equal_Version#01 (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV#01 (0.00s) PASS ok k8s.io/kops/pkg/k8sversion 0.052s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.00s) === RUN TestToVersionedJSON --- PASS: TestToVersionedJSON (0.00s) === RUN TestRewriteAPIGroup --- PASS: TestRewriteAPIGroup (0.00s) PASS ok k8s.io/kops/pkg/kopscodecs 0.026s === RUN TestBuildKubecfg === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster === RUN TestBuildKubecfg/Public_DNS_with_kops_auth_plugin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port --- PASS: TestBuildKubecfg (0.14s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster (0.01s) --- PASS: TestBuildKubecfg/Public_DNS_with_kops_auth_plugin (0.01s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port (0.03s) PASS ok k8s.io/kops/pkg/kubeconfig 0.199s ? k8s.io/kops/pkg/kubemanifest [no test files] === RUN Test_ProxyFunc --- PASS: Test_ProxyFunc (0.00s) === RUN TestBootstrapUserData --- PASS: TestBootstrapUserData (0.01s) === RUN TestValidateAWSVolumeAllow50ratio --- PASS: TestValidateAWSVolumeAllow50ratio (0.00s) PASS ok k8s.io/kops/pkg/model 0.328s === RUN TestRootVolumeOptimizationFlag --- PASS: TestRootVolumeOptimizationFlag (0.00s) === RUN TestAPIServerAdditionalSecurityGroupsWithNLB === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Master === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Node --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Master (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Node (0.00s) === RUN TestJoinSuffixes --- PASS: TestJoinSuffixes (0.00s) === RUN TestIAMServiceEC2 --- PASS: TestIAMServiceEC2 (0.00s) === RUN Test_formatAWSIAMStatement === RUN Test_formatAWSIAMStatement/namespace_and_name_without_wildcard === RUN Test_formatAWSIAMStatement/name_contains_wildcard === RUN Test_formatAWSIAMStatement/namespace_contains_wildcard --- PASS: Test_formatAWSIAMStatement (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_and_name_without_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/name_contains_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_contains_wildcard (0.00s) PASS ok k8s.io/kops/pkg/model/awsmodel 0.061s === RUN TestAPILoadBalancerModelBuilder_Build --- PASS: TestAPILoadBalancerModelBuilder_Build (0.00s) === RUN TestSubnetForLoadbalancer --- PASS: TestSubnetForLoadbalancer (0.00s) === RUN TestCloudTagsForInstanceGroup --- PASS: TestCloudTagsForInstanceGroup (0.00s) === RUN TestNetworkModelBuilder_Build --- PASS: TestNetworkModelBuilder_Build (0.00s) === RUN TestResourceGroupModelBuilder_Build --- PASS: TestResourceGroupModelBuilder_Build (0.00s) === RUN TestVMScaleSetModelBuilder_Build --- PASS: TestVMScaleSetModelBuilder_Build (0.00s) === RUN TestGetCapacity === RUN TestGetCapacity/test_case_0 === RUN TestGetCapacity/test_case_1 === RUN TestGetCapacity/test_case_2 === RUN TestGetCapacity/test_case_3 --- PASS: TestGetCapacity (0.00s) --- PASS: TestGetCapacity/test_case_0 (0.00s) --- PASS: TestGetCapacity/test_case_1 (0.00s) --- PASS: TestGetCapacity/test_case_2 (0.00s) --- PASS: TestGetCapacity/test_case_3 (0.00s) === RUN TestGetStorageProfile === RUN TestGetStorageProfile/test_case_0 === RUN TestGetStorageProfile/test_case_1 --- PASS: TestGetStorageProfile (0.00s) --- PASS: TestGetStorageProfile/test_case_0 (0.00s) --- PASS: TestGetStorageProfile/test_case_1 (0.00s) === RUN TestParseImage === RUN TestParseImage/test_case_0 === RUN TestParseImage/test_case_1 === RUN TestParseImage/test_case_2 === RUN TestParseImage/test_case_3 --- PASS: TestParseImage (0.00s) --- PASS: TestParseImage/test_case_0 (0.00s) --- PASS: TestParseImage/test_case_1 (0.00s) --- PASS: TestParseImage/test_case_2 (0.00s) --- PASS: TestParseImage/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/model/azuremodel 0.097s === RUN TestCloudConfigurationOptionsBuilder === RUN TestCloudConfigurationOptionsBuilder/neither === RUN TestCloudConfigurationOptionsBuilder/all_false === RUN TestCloudConfigurationOptionsBuilder/all_true === RUN TestCloudConfigurationOptionsBuilder/os_false === RUN TestCloudConfigurationOptionsBuilder/os_true === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_true === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_true --- PASS: TestCloudConfigurationOptionsBuilder (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/neither (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_true (0.00s) === RUN Test_Build_Containerd_Supported_Version --- PASS: Test_Build_Containerd_Supported_Version (0.00s) === RUN Test_Build_Containerd_Unneeded_Runtime --- PASS: Test_Build_Containerd_Unneeded_Runtime (0.00s) === RUN Test_Build_Containerd_Needed_Runtime --- PASS: Test_Build_Containerd_Needed_Runtime (0.00s) === RUN TestImage I0811 05:31:44.444242 38470 featureflag.go:162] FeatureFlag "ImageDigest"=false --- PASS: TestImage (0.00s) === RUN Test_Build_KCM_Builder --- PASS: Test_Build_KCM_Builder (0.00s) === RUN Test_Build_KCM_Builder_Change_Duration --- PASS: Test_Build_KCM_Builder_Change_Duration (0.00s) === RUN Test_Build_KCM_Builder_CIDR_Mask_Size === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: (0.00s) === RUN TestFeatureGatesKubernetesVersion --- PASS: TestFeatureGatesKubernetesVersion (0.00s) === RUN TestFeatureGatesOverride --- PASS: TestFeatureGatesOverride (0.00s) === RUN Test_Build_Scheduler_Without_PolicyConfigMap --- PASS: Test_Build_Scheduler_Without_PolicyConfigMap (0.00s) === RUN Test_Build_Scheduler_PolicyConfigMap_Supported_Version --- PASS: Test_Build_Scheduler_PolicyConfigMap_Supported_Version (0.00s) PASS ok k8s.io/kops/pkg/model/components 0.084s ? k8s.io/kops/pkg/model/components/addonmanifests [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awscloudcontrollermanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsebscsidriver [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsloadbalancercontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/clusterautoscaler [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/dnscontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/externaldns [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/karpenter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/kuberouter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/nodeterminationhandler [no test files] === RUN Test_RunEtcdManagerBuilder I0811 05:31:52.001440 39886 featureflag.go:162] FeatureFlag "ImageDigest"=false === RUN Test_RunEtcdManagerBuilder/basedir=tests/minimal === RUN Test_RunEtcdManagerBuilder/basedir=tests/interval === RUN Test_RunEtcdManagerBuilder/basedir=tests/proxy === RUN Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings W0811 05:31:52.030170 39886 model.go:347] overriding log level in manifest (embedded etcd manifest), new level is 3 W0811 05:31:52.030322 39886 model.go:484] overloading ENV var in manifest (embedded etcd manifest) with ETCD_QUOTA_BACKEND_BYTES=10737418240 W0811 05:31:52.033268 39886 model.go:347] overriding log level in manifest (embedded etcd manifest), new level is 3 W0811 05:31:52.033415 39886 model.go:484] overloading ENV var in manifest (embedded etcd manifest) with ETCD_QUOTA_BACKEND_BYTES=10737418240 --- PASS: Test_RunEtcdManagerBuilder (0.04s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/minimal (0.02s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/interval (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/proxy (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings (0.01s) PASS ok k8s.io/kops/pkg/model/components/etcdmanager 0.093s ? k8s.io/kops/pkg/model/components/kopscontroller [no test files] === RUN Test_RunKubeApiserverBuilder I0811 05:31:51.903815 39837 featureflag.go:162] FeatureFlag "ImageDigest"=false === RUN Test_RunKubeApiserverBuilder/basedir=tests/minimal --- PASS: Test_RunKubeApiserverBuilder (0.02s) --- PASS: Test_RunKubeApiserverBuilder/basedir=tests/minimal (0.02s) PASS ok k8s.io/kops/pkg/model/components/kubeapiserver 0.345s === RUN TestDefaultInstanceGroupVolumeSize --- PASS: TestDefaultInstanceGroupVolumeSize (0.00s) PASS ok k8s.io/kops/pkg/model/defaults 0.086s ? k8s.io/kops/pkg/model/domodel [no test files] ? k8s.io/kops/pkg/model/gcemodel [no test files] ? k8s.io/kops/pkg/model/hetznermodel [no test files] === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) === RUN TestPolicyGeneration --- PASS: TestPolicyGeneration (0.00s) === RUN TestEmptyPolicy --- PASS: TestEmptyPolicy (0.00s) PASS ok k8s.io/kops/pkg/model/iam 0.086s === RUN TestServerGroupBuilder I0811 05:31:52.423063 39906 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:31:52.440225 39906 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:31:52.453722 39906 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:31:52.473220 39906 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:31:52.533572 39906 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:31:52.550688 39906 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:31:52.570122 39906 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:31:52.579629 39906 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:31:52.588416 39906 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:31:52.678500 39906 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:31:52.686084 39906 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:31:52.692454 39906 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:31:52.698128 39906 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:31:52.704297 39906 featureflag.go:162] FeatureFlag "ImageDigest"=false --- PASS: TestServerGroupBuilder (0.29s) PASS ok k8s.io/kops/pkg/model/openstackmodel 0.364s === RUN Test_NodeUpTabs --- PASS: Test_NodeUpTabs (0.00s) PASS ok k8s.io/kops/pkg/model/resources 0.023s ? k8s.io/kops/pkg/nodeidentity [no test files] ? k8s.io/kops/pkg/nodeidentity/aws [no test files] === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetVMSSNameFromProviderID === RUN TestGetVMSSNameFromProviderID/test_case_0 === RUN TestGetVMSSNameFromProviderID/test_case_1 === RUN TestGetVMSSNameFromProviderID/test_case_2 --- PASS: TestGetVMSSNameFromProviderID (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_0 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_1 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_2 (0.00s) === RUN TestIdentifyNode === RUN TestIdentifyNode/test_case_0 === RUN TestIdentifyNode/test_case_1 --- PASS: TestIdentifyNode (0.00s) --- PASS: TestIdentifyNode/test_case_0 (0.00s) --- PASS: TestIdentifyNode/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/nodeidentity/azure 0.008s ? k8s.io/kops/pkg/nodeidentity/do [no test files] ? k8s.io/kops/pkg/nodeidentity/gce [no test files] ? k8s.io/kops/pkg/nodeidentity/hetzner [no test files] ? k8s.io/kops/pkg/nodeidentity/openstack [no test files] === RUN TestBuildNodeLabels === RUN TestBuildNodeLabels/RoleControlPlane === RUN TestBuildNodeLabels/RoleNode --- PASS: TestBuildNodeLabels (0.00s) --- PASS: TestBuildNodeLabels/RoleControlPlane (0.00s) --- PASS: TestBuildNodeLabels/RoleNode (0.00s) PASS ok k8s.io/kops/pkg/nodelabels 0.082s === RUN TestGenerateCertificate === RUN TestGenerateCertificate/selfsigned === RUN TestGenerateCertificate/client === RUN TestGenerateCertificate/server --- PASS: TestGenerateCertificate (0.01s) --- PASS: TestGenerateCertificate/selfsigned (0.01s) --- PASS: TestGenerateCertificate/client (0.00s) --- PASS: TestGenerateCertificate/server (0.00s) === RUN TestCertificateRoundTrip --- PASS: TestCertificateRoundTrip (0.00s) === RUN TestIssueCert === RUN TestIssueCert/ca === RUN TestIssueCert/client === RUN TestIssueCert/clientOneYear === RUN TestIssueCert/#00 === RUN TestIssueCert/server --- PASS: TestIssueCert (1.77s) --- PASS: TestIssueCert/ca (0.50s) --- PASS: TestIssueCert/client (0.23s) --- PASS: TestIssueCert/clientOneYear (1.03s) --- PASS: TestIssueCert/#00 (0.00s) --- PASS: TestIssueCert/server (0.00s) === RUN TestPrivateKeyRoundTripRSA === RUN TestPrivateKeyRoundTripRSA/rsa === RUN TestPrivateKeyRoundTripRSA/ecdsa --- PASS: TestPrivateKeyRoundTripRSA (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/rsa (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/ecdsa (0.00s) === RUN Test_AWSFingerprint_RsaKey1 --- PASS: Test_AWSFingerprint_RsaKey1 (0.00s) === RUN Test_AWSFingerprint_RsaKeyEncrypted --- PASS: Test_AWSFingerprint_RsaKeyEncrypted (0.00s) === RUN Test_AWSFingerprint_Ed25519Key --- PASS: Test_AWSFingerprint_Ed25519Key (0.00s) === RUN Test_AWSFingerprint_TrickyWhitespace --- PASS: Test_AWSFingerprint_TrickyWhitespace (0.00s) === RUN Test_AWSFingerprint_DsaKey --- PASS: Test_AWSFingerprint_DsaKey (0.00s) === RUN Test_OpenSSHFingerprint_RsaKey1 --- PASS: Test_OpenSSHFingerprint_RsaKey1 (0.00s) PASS ok k8s.io/kops/pkg/pki 1.790s === RUN TestLongDesc --- PASS: TestLongDesc (0.00s) PASS ok k8s.io/kops/pkg/pretty 0.082s ? k8s.io/kops/pkg/rbac [no test files] ? k8s.io/kops/pkg/resolver [no test files] ? k8s.io/kops/pkg/resources [no test files] === RUN TestAddUntaggedRouteTables I0811 05:31:48.650709 39137 routetable.go:53] DescribeRouteTables: { } W0811 05:31:48.650889 39137 tags.go:67] cluster tag not found on route-table:rtb-1234 I0811 05:31:48.650911 39137 aws.go:225] Skipping route table in VPC, but with wrong cluster tag ("other.example.com") --- PASS: TestAddUntaggedRouteTables (0.00s) === RUN TestListRouteTables I0811 05:31:48.651039 39137 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["me.example.com"] }] } I0811 05:31:48.651172 39137 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/me.example.com"] }] } --- PASS: TestListRouteTables (0.00s) === RUN TestSharedVolume I0811 05:31:48.651303 39137 volumes.go:33] CreateVolume: { TagSpecifications: [{ ResourceType: "volume", Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "shared" }] }] } I0811 05:31:48.651437 39137 volumes.go:33] CreateVolume: { TagSpecifications: [{ Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "owned" }] }] } I0811 05:31:48.651488 39137 volumes.go:128] DescribeVolumes: { } W0811 05:31:48.651540 39137 aws.go:2239] (new) cluster tag not found on volume:vol-2 aws_test.go:192: 2 --- PASS: TestSharedVolume (0.00s) === RUN TestMatchesElbTags --- PASS: TestMatchesElbTags (0.00s) === RUN TestListVPCs I0811 05:31:48.651726 39137 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:31:48.651988 39137 internetgateways.go:66] CreateInternetGateway: { } I0811 05:31:48.652493 39137 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:31:48.652584 39137 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:31:48.652665 39137 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:31:48.652728 39137 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:31:48.652761 39137 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:31:48.652809 39137 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:31:48.652860 39137 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:31:48.652909 39137 tags.go:42] CreateTags { Resources: ["vpc-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "legacy.example.com" }] } I0811 05:31:48.652954 39137 tags.go:42] CreateTags { Resources: ["vpc-shared"], Tags: [{ Key: "kubernetes.io/cluster/shared.example.com", Value: "shared" }] } I0811 05:31:48.652994 39137 tags.go:42] CreateTags { Resources: ["vpc-shared-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "shared-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/shared-with-legacy.example.com", Value: "shared" }] } I0811 05:31:48.653059 39137 tags.go:42] CreateTags { Resources: ["vpc-owned"], Tags: [{ Key: "kubernetes.io/cluster/owned.example.com", Value: "owned" }] } I0811 05:31:48.653096 39137 tags.go:42] CreateTags { Resources: ["vpc-owned-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "owned-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/owned-with-legacy.example.com", Value: "owned" }] } I0811 05:31:48.653147 39137 tags.go:42] CreateTags { Resources: ["vpc-other"], Tags: [{ Key: "KubernetesCluster", Value: "other.example.com" },{ Key: "kubernetes.io/cluster/other.example.com", Value: "shared" }] } I0811 05:31:48.653203 39137 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["mismatch.example.com"] }] } I0811 05:31:48.653257 39137 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/mismatch.example.com"] }] } I0811 05:31:48.653298 39137 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["legacy.example.com"] }] } I0811 05:31:48.653335 39137 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/legacy.example.com"] }] } I0811 05:31:48.653378 39137 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared-with-legacy.example.com"] }] } I0811 05:31:48.653416 39137 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared-with-legacy.example.com"] }] } I0811 05:31:48.653474 39137 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared.example.com"] }] } I0811 05:31:48.653512 39137 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared.example.com"] }] } I0811 05:31:48.653581 39137 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned-with-legacy.example.com"] }] } I0811 05:31:48.653622 39137 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned-with-legacy.example.com"] }] } I0811 05:31:48.653667 39137 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned.example.com"] }] } I0811 05:31:48.653705 39137 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned.example.com"] }] } --- PASS: TestListVPCs (0.00s) PASS ok k8s.io/kops/pkg/resources/aws 0.072s === RUN TestListResourcesAzure --- PASS: TestListResourcesAzure (0.00s) === RUN TestIsOwnedByCluster === RUN TestIsOwnedByCluster/test_case_0 === RUN TestIsOwnedByCluster/test_case_1 === RUN TestIsOwnedByCluster/test_case_2 === RUN TestIsOwnedByCluster/test_case_3 --- PASS: TestIsOwnedByCluster (0.00s) --- PASS: TestIsOwnedByCluster/test_case_0 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_1 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_2 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/resources/azure 0.026s ? k8s.io/kops/pkg/resources/digitalocean [no test files] === RUN TestNameMatch --- PASS: TestNameMatch (0.00s) === RUN TestMatchesClusterNameWithUUID --- PASS: TestMatchesClusterNameWithUUID (0.00s) PASS ok k8s.io/kops/pkg/resources/gce 0.111s ? k8s.io/kops/pkg/resources/hetzner [no test files] ? k8s.io/kops/pkg/resources/openstack [no test files] ? k8s.io/kops/pkg/resources/ops [no test files] ? k8s.io/kops/pkg/resources/spotinst [no test files] ? k8s.io/kops/pkg/sshcredentials [no test files] === RUN TestEscapeCommand --- PASS: TestEscapeCommand (0.00s) === RUN TestRawManifest --- PASS: TestRawManifest (0.00s) === RUN TestRawMixedManifest --- PASS: TestRawMixedManifest (0.00s) === RUN TestKeyPairOnlyManifest --- PASS: TestKeyPairOnlyManifest (0.00s) === RUN TestUnitFileExtensionValid === RUN TestUnitFileExtensionValid/valid_extension:_.automount === RUN TestUnitFileExtensionValid/valid_extension:_.device === RUN TestUnitFileExtensionValid/valid_extension:_.mount === RUN TestUnitFileExtensionValid/valid_extension:_.path === RUN TestUnitFileExtensionValid/valid_extension:_.scope === RUN TestUnitFileExtensionValid/valid_extension:_.service === RUN TestUnitFileExtensionValid/valid_extension:_.slice === RUN TestUnitFileExtensionValid/valid_extension:_.socket === RUN TestUnitFileExtensionValid/valid_extension:_.swap === RUN TestUnitFileExtensionValid/valid_extension:_.target === RUN TestUnitFileExtensionValid/valid_extension:_.timer === RUN TestUnitFileExtensionValid/invalid_extension:_(no_extension) === RUN TestUnitFileExtensionValid/invalid_extension:_.not-valid --- PASS: TestUnitFileExtensionValid (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.automount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.device (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.mount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.path (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.scope (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.service (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.slice (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.socket (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.swap (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.target (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.timer (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_(no_extension) (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_.not-valid (0.00s) PASS ok k8s.io/kops/pkg/systemd 0.079s ? k8s.io/kops/pkg/templates [no test files] ? k8s.io/kops/pkg/testutils [no test files] ? k8s.io/kops/pkg/testutils/golden [no test files] ? k8s.io/kops/pkg/tokens [no test files] === RUN TestTruncateString === RUN TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false --- PASS: TestTruncateString (0.00s) --- PASS: TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false (0.00s) PASS ok k8s.io/kops/pkg/truncate 0.060s ? k8s.io/kops/pkg/try [no test files] === RUN TestJoin --- PASS: TestJoin (0.00s) PASS ok k8s.io/kops/pkg/urls 0.035s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) PASS ok k8s.io/kops/pkg/util/stringorslice 0.066s === RUN Test_Overlap --- PASS: Test_Overlap (0.00s) === RUN Test_BelongsTo --- PASS: Test_BelongsTo (0.00s) === RUN Test_SplitInto8 --- PASS: Test_SplitInto8 (0.00s) PASS ok k8s.io/kops/pkg/util/subnet 0.079s === RUN TestRenderGeneralOK --- PASS: TestRenderGeneralOK (0.01s) === RUN TestRenderMissingValue --- PASS: TestRenderMissingValue (0.02s) === RUN TestRenderIndent --- PASS: TestRenderIndent (0.00s) === RUN TestRenderChannelFunctions --- PASS: TestRenderChannelFunctions (0.00s) === RUN TestRenderSnippet --- PASS: TestRenderSnippet (0.00s) === RUN TestRenderContext --- PASS: TestRenderContext (0.00s) === RUN TestAllowForMissingVars --- PASS: TestAllowForMissingVars (0.00s) === RUN TestRenderIntegration --- PASS: TestRenderIntegration (0.00s) PASS ok k8s.io/kops/pkg/util/templater 0.110s === RUN Test_ValidateCloudGroupMissing --- PASS: Test_ValidateCloudGroupMissing (0.00s) === RUN Test_ValidateNodesNotEnough --- PASS: Test_ValidateNodesNotEnough (0.00s) === RUN Test_ValidateDetachedNodesDontCount --- PASS: Test_ValidateDetachedNodesDontCount (0.00s) === RUN Test_ValidateNodeNotReady --- PASS: Test_ValidateNodeNotReady (0.00s) === RUN Test_ValidateMastersNotEnough --- PASS: Test_ValidateMastersNotEnough (0.00s) === RUN Test_ValidateMasterNotReady --- PASS: Test_ValidateMasterNotReady (0.00s) === RUN Test_ValidateMasterStaticPods --- PASS: Test_ValidateMasterStaticPods (0.00s) === RUN Test_ValidateNoPodFailures --- PASS: Test_ValidateNoPodFailures (0.00s) === RUN Test_ValidatePodFailure === RUN Test_ValidatePodFailure/pending-node-kube-system === RUN Test_ValidatePodFailure/pending-node-kube-system#01 === RUN Test_ValidatePodFailure/pending-node-kube-system#02 === RUN Test_ValidatePodFailure/pending-node-otherNamespace === RUN Test_ValidatePodFailure/pending-node-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-node-otherNamespace#02 === RUN Test_ValidatePodFailure/pending-cluster-kube-system === RUN Test_ValidatePodFailure/pending-cluster-kube-system#01 === RUN Test_ValidatePodFailure/pending-cluster-kube-system#02 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-node-kube-system === RUN Test_ValidatePodFailure/notready-node-kube-system#01 === RUN Test_ValidatePodFailure/notready-node-kube-system#02 === RUN Test_ValidatePodFailure/notready-node-otherNamespace === RUN Test_ValidatePodFailure/notready-node-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-node-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-cluster-kube-system === RUN Test_ValidatePodFailure/notready-cluster-kube-system#01 === RUN Test_ValidatePodFailure/notready-cluster-kube-system#02 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-node-kube-system === RUN Test_ValidatePodFailure/unknown-node-kube-system#01 === RUN Test_ValidatePodFailure/unknown-node-kube-system#02 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#01 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#02 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 --- PASS: Test_ValidatePodFailure (0.01s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 (0.00s) === RUN Test_ValidateBastionNodes === RUN Test_ValidateBastionNodes/instancegroup's_nodes_not_ready === RUN Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready --- PASS: Test_ValidateBastionNodes (0.00s) --- PASS: Test_ValidateBastionNodes/instancegroup's_nodes_not_ready (0.00s) --- PASS: Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready (0.00s) === RUN Test_ValidateDetachedNodesNotValidated --- PASS: Test_ValidateDetachedNodesNotValidated (0.00s) PASS ok k8s.io/kops/pkg/validation 0.072s ? k8s.io/kops/pkg/values [no test files] ? k8s.io/kops/pkg/wellknownoperators [no test files] ? k8s.io/kops/pkg/wellknownports [no test files] ? k8s.io/kops/pkg/wellknownusers [no test files] ? k8s.io/kops/pkg/zones [no test files] ? k8s.io/kops/protokube/cmd/protokube [no test files] ? k8s.io/kops/protokube/pkg/etcd [no test files] ? k8s.io/kops/protokube/pkg/gossip [no test files] ? k8s.io/kops/protokube/pkg/gossip/aws [no test files] === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetInternalIP --- PASS: TestGetInternalIP (0.00s) === RUN TestGetSeeds --- PASS: TestGetSeeds (0.00s) PASS ok k8s.io/kops/protokube/pkg/gossip/azure 0.004s ? k8s.io/kops/protokube/pkg/gossip/dns [no test files] === RUN TestRemovesDuplicateGuardedBlocks W0811 05:30:57.527526 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.527603 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.527621 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.527637 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.527649 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.527663 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.527678 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.527693 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.527707 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.527721 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.527734 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.527749 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.527769 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.527785 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.527800 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.527816 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.530356 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.530393 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.530489 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.530508 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.530596 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.530615 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.530698 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.530716 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.530830 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.530850 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.530950 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.530970 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.531056 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.531077 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.531169 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.531187 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.531271 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.531289 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.531370 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.531388 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.531503 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.531522 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.531605 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.531623 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.531703 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.531721 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.531817 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.531835 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.531921 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.531940 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.532027 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.532044 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.532122 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.532139 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.532220 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.532238 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.532317 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.532334 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.532483 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.532506 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.532586 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.532608 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.532694 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.532715 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.532798 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.532818 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.532900 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.532919 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.532999 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.533036 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.533121 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.533141 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.533217 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.533238 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.533316 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.533335 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.533413 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.533433 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.533510 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.533531 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.533621 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.533642 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.533737 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.533764 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.533848 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.533895 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.533983 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.534000 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.534079 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.534097 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.534190 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.534210 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.534301 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.534319 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.534413 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.534431 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.534539 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.534558 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.534656 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.534673 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.534779 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.534797 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.534874 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.534892 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.534975 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.535000 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.535070 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.535087 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.535165 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.535182 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.535258 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.535278 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.535356 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.535374 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.535458 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.535476 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.535549 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.535565 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.535648 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.535667 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.536232 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.536257 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.536340 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.536360 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.536447 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.536467 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.536547 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.536566 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.536641 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.536661 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.536739 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.536758 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.536834 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.536873 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.536956 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.536977 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.537057 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.537078 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.537194 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.537215 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.537289 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.537308 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.537390 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.537411 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.537520 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.537541 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.537624 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.537643 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.537724 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.537743 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.537823 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.537842 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.537917 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.537936 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.538041 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.538064 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.538147 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.538168 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.538256 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.538277 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.538355 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.538374 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.538475 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.538496 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.538577 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.538598 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.538680 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.538700 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.538779 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.538798 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.538871 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.538890 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.538963 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.538984 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.539075 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.539097 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.539182 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.539204 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.539313 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.540150 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.540310 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.540335 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.540426 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.540448 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.540526 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.540545 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.540671 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.540696 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.540771 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.540789 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.540902 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.540924 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.541004 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.541027 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.541112 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.541133 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.541212 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.541232 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.541341 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.541361 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.541437 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.541455 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.541533 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.541555 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.541639 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.541661 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.541742 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.541837 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.541959 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.541983 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.542068 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.542121 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.542234 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.542257 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.542336 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.542372 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.542459 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.542480 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRemovesDuplicateGuardedBlocks (0.02s) === RUN TestRecoversFromBadNesting W0811 05:30:57.543320 16676 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0811 05:30:57.543339 16676 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0811 05:30:57.543358 16676 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0811 05:30:57.543373 16676 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0811 05:30:57.543387 16676 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0811 05:30:57.543398 16676 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0811 05:30:57.543408 16676 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0811 05:30:57.543444 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.543462 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.543475 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.543487 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.543502 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.543515 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.543529 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.543543 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.543556 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.543568 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.543584 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.543596 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.543610 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.543624 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.543637 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.543652 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.552275 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.552325 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.552443 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.552467 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.552557 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.552578 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.552678 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.552700 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.552832 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.552853 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.552938 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.552960 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.553047 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.553073 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.553151 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.553171 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.553251 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.553271 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.553350 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.553370 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.553448 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.553473 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.553555 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.553575 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.553673 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.553694 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.553779 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.553801 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.553883 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.553904 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.553983 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.554003 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.554079 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.554101 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.554197 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.554217 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.554344 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.554365 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.554452 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.554473 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.554591 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.554612 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.554689 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.554709 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.554797 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.554867 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.554980 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.555001 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.555076 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.555097 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.555182 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.555202 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.555295 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.555318 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.555401 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.555422 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.555538 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.555557 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.555631 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.555674 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.555766 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.555786 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.555864 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.555920 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.556003 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.556021 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.556098 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.556116 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.556205 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.556223 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.556305 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.556324 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.556404 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.556421 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.556501 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.556519 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.556599 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.556619 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.556696 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.556713 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.556792 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.556809 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.556901 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.556919 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.557017 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.557037 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.557124 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.557143 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.557222 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.557240 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.557321 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.557338 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.557414 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.557431 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.557510 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.557528 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.557607 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.557625 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.557706 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.557724 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.557815 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.557833 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.557920 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.557938 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.558051 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.558070 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.558153 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.558171 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.558246 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.558263 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.558343 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.558361 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.558443 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.558461 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.558536 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.558556 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.558637 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.558654 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.558732 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.558749 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.558832 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.558855 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.558935 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.558953 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.559042 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.559061 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.559138 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.559155 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.559233 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.559251 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.559327 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.559348 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.559429 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.559448 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.559531 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.559549 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.559628 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.559646 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.559720 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.559737 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.559814 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.559834 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.559915 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.559932 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.560008 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.560026 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.560121 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.560141 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.560217 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.560235 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.560317 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.560337 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.560454 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.560472 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.560557 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.560574 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.560653 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.560672 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.560748 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.560764 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.560835 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.560850 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.560923 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.560938 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.561006 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.561022 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.561093 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.561111 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.561183 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.561198 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.561269 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.561285 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.561355 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.561371 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.561440 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.561460 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.561533 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.561550 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.561629 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.561645 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.561728 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.561745 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.561819 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.561838 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.561915 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.561931 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.562004 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.562020 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.562095 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.562111 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.562186 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.562204 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.562300 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.562318 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.562419 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.562439 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0811 05:30:57.562514 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0811 05:30:57.562531 16676 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRecoversFromBadNesting (0.02s) PASS ok k8s.io/kops/protokube/pkg/gossip/dns/hosts 0.082s ? k8s.io/kops/protokube/pkg/gossip/dns/provider [no test files] ? k8s.io/kops/protokube/pkg/gossip/do [no test files] ? k8s.io/kops/protokube/pkg/gossip/hetzner [no test files] ? k8s.io/kops/protokube/pkg/gossip/memberlist [no test files] ? k8s.io/kops/protokube/pkg/gossip/mesh [no test files] ? k8s.io/kops/protokube/pkg/gossip/openstack [no test files] ? k8s.io/kops/protokube/pkg/hostmount [no test files] ? k8s.io/kops/protokube/pkg/protokube [no test files] === RUN TestSerializeEmptyCluster --- PASS: TestSerializeEmptyCluster (0.00s) PASS ok k8s.io/kops/tests/codecs 0.026s ? k8s.io/kops/tests/fuzz [no test files] === RUN TestKopsUpgrades --- PASS: TestKopsUpgrades (0.01s) === RUN TestKubernetesUpgrades --- PASS: TestKubernetesUpgrades (0.00s) === RUN TestFindImage --- PASS: TestFindImage (0.00s) === RUN TestRecommendedKubernetesVersion --- PASS: TestRecommendedKubernetesVersion (0.00s) === RUN TestOrdering --- PASS: TestOrdering (0.00s) === RUN TestChannelsSelfConsistent --- PASS: TestChannelsSelfConsistent (0.05s) === RUN TestChannelImages === RUN TestChannelImages/stable-channel === RUN TestChannelImages/alpha-channel --- PASS: TestChannelImages (0.00s) --- PASS: TestChannelImages/stable-channel (0.00s) --- PASS: TestChannelImages/alpha-channel (0.00s) PASS ok k8s.io/kops/tests/integration/channel 0.187s ? k8s.io/kops/tests/integration/channel/simple [no test files] === RUN TestConversionMinimal === RUN TestConversionMinimal/legacy-v1alpha2-v1alpha2 --- PASS: TestConversionMinimal (0.05s) --- PASS: TestConversionMinimal/legacy-v1alpha2-v1alpha2 (0.05s) PASS ok k8s.io/kops/tests/integration/conversion 0.077s ? k8s.io/kops/upup/models [no test files] === RUN Test_tryResourceAsString --- PASS: Test_tryResourceAsString (0.00s) === RUN Test_DryrunTarget_PrintReport --- PASS: Test_DryrunTarget_PrintReport (0.00s) === RUN TestWriteFile I0811 05:31:32.017503 34777 files.go:57] Writing file "/tmp/TestWriteFile467835140/001/SubDir/test1.tmp" I0811 05:31:32.017636 34777 files.go:113] Changing file mode for "/tmp/TestWriteFile467835140/001/SubDir/test1.tmp" to -rw-r--r-- --- PASS: TestWriteFile (0.00s) === RUN TestBigInt_Format --- PASS: TestBigInt_Format (0.00s) === RUN TestVFSCAStoreRoundTrip --- PASS: TestVFSCAStoreRoundTrip (0.00s) === RUN TestVFSCAStoreRoundTripWithVault vfs_castore_test.go:161: No vault dev token set --- SKIP: TestVFSCAStoreRoundTripWithVault (0.00s) === RUN TestAddItem === RUN TestAddItem/first_primary === RUN TestAddItem/first_secondary === RUN TestAddItem/firstBigSerial_primary === RUN TestAddItem/firstBigSerial_secondary === RUN TestAddItem/after_primary === RUN TestAddItem/after_secondary === RUN TestAddItem/bigSerialAfter_primary === RUN TestAddItem/bigSerialAfter_secondary === RUN TestAddItem/before_primary === RUN TestAddItem/before_secondary === RUN TestAddItem/first_certonly_primary === RUN TestAddItem/first_certonly_secondary === RUN TestAddItem/after_certonly_primary === RUN TestAddItem/after_certonly_secondary === RUN TestAddItem/before_certonly_primary === RUN TestAddItem/before_certonly_secondary --- PASS: TestAddItem (0.00s) --- PASS: TestAddItem/first_primary (0.00s) --- PASS: TestAddItem/first_secondary (0.00s) --- PASS: TestAddItem/firstBigSerial_primary (0.00s) --- PASS: TestAddItem/firstBigSerial_secondary (0.00s) --- PASS: TestAddItem/after_primary (0.00s) --- PASS: TestAddItem/after_secondary (0.00s) --- PASS: TestAddItem/bigSerialAfter_primary (0.00s) --- PASS: TestAddItem/bigSerialAfter_secondary (0.00s) --- PASS: TestAddItem/before_primary (0.00s) --- PASS: TestAddItem/before_secondary (0.00s) --- PASS: TestAddItem/first_certonly_primary (0.00s) --- PASS: TestAddItem/first_certonly_secondary (0.00s) --- PASS: TestAddItem/after_certonly_primary (0.00s) --- PASS: TestAddItem/after_certonly_secondary (0.00s) --- PASS: TestAddItem/before_certonly_primary (0.00s) --- PASS: TestAddItem/before_certonly_secondary (0.00s) PASS ok k8s.io/kops/upup/pkg/fi 0.028s === RUN TestBootstrapChannelBuilder_BuildTasks I0811 05:31:58.977721 40458 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:31:58.977966 40458 internetgateways.go:66] CreateInternetGateway: { } I0811 05:31:58.978018 40458 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:31:58.978055 40458 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:31:58.978106 40458 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:31:58.978146 40458 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:31:58.978163 40458 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:31:58.978194 40458 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:31:58.978222 40458 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:31:58.987626 40458 zones.go:93] ListHostedZonesPages { } I0811 05:31:59.034715 40458 zones.go:93] ListHostedZonesPages { } I0811 05:31:59.139403 40458 zones.go:93] ListHostedZonesPages { } I0811 05:31:59.195969 40458 zones.go:93] ListHostedZonesPages { } I0811 05:31:59.255505 40458 zones.go:93] ListHostedZonesPages { } I0811 05:31:59.325156 40458 zones.go:93] ListHostedZonesPages { } I0811 05:31:59.379996 40458 zones.go:93] ListHostedZonesPages { } I0811 05:31:59.436451 40458 zones.go:93] ListHostedZonesPages { } I0811 05:31:59.544460 40458 zones.go:93] ListHostedZonesPages { } I0811 05:31:59.938459 40458 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_BuildTasks (1.01s) === RUN TestBootstrapChannelBuilder_ServiceAccountIAM I0811 05:31:59.984777 40458 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:31:59.984931 40458 internetgateways.go:66] CreateInternetGateway: { } I0811 05:31:59.984986 40458 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:31:59.985020 40458 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:31:59.985052 40458 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:31:59.985080 40458 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:31:59.985096 40458 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:31:59.985121 40458 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:31:59.985139 40458 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:31:59.985158 40458 featureflag.go:165] Unknown FeatureFlag "+UseServiceAccountExternalPermissions" I0811 05:31:59.986376 40458 zones.go:93] ListHostedZonesPages { } I0811 05:32:00.053224 40458 featureflag.go:165] Unknown FeatureFlag "-UseServiceAccountExternalPermissions" --- PASS: TestBootstrapChannelBuilder_ServiceAccountIAM (0.07s) === RUN TestBootstrapChannelBuilder_AWSCloudController I0811 05:32:00.053568 40458 featureflag.go:162] FeatureFlag "ImageDigest"=false I0811 05:32:00.053826 40458 internetgateways.go:66] CreateInternetGateway: { } I0811 05:32:00.053908 40458 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0811 05:32:00.053965 40458 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0811 05:32:00.054007 40458 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0811 05:32:00.054043 40458 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0811 05:32:00.054066 40458 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0811 05:32:00.054096 40458 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0811 05:32:00.054120 40458 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0811 05:32:00.056078 40458 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_AWSCloudController (0.11s) === RUN TestContainerdVersionUrlHash === RUN TestContainerdVersionUrlHash/1.3.4-amd64 === RUN TestContainerdVersionUrlHash/1.3.4-arm64 === RUN TestContainerdVersionUrlHash/1.3.10-amd64 === RUN TestContainerdVersionUrlHash/1.3.10-arm64 === RUN TestContainerdVersionUrlHash/1.4.1-amd64 === RUN TestContainerdVersionUrlHash/1.4.1-arm64 === RUN TestContainerdVersionUrlHash/1.4.4-amd64 === RUN TestContainerdVersionUrlHash/1.4.4-arm64 --- PASS: TestContainerdVersionUrlHash (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-arm64 (0.00s) === RUN TestContainerdVersionUrl === RUN TestContainerdVersionUrl/1.4.1- === RUN TestContainerdVersionUrl/1.4.1-arm === RUN TestContainerdVersionUrl/-amd64 === RUN TestContainerdVersionUrl/-arm64 === RUN TestContainerdVersionUrl/1.1.1-amd64 === RUN TestContainerdVersionUrl/1.1.1-arm64 === RUN TestContainerdVersionUrl/1.3.5-amd64 === RUN TestContainerdVersionUrl/1.3.5-arm64 === RUN TestContainerdVersionUrl/1.3.4-amd64 === RUN TestContainerdVersionUrl/1.3.4-arm64 === RUN TestContainerdVersionUrl/1.4.1-amd64 === RUN TestContainerdVersionUrl/1.4.1-arm64 === RUN TestContainerdVersionUrl/1.4.3-amd64 === RUN TestContainerdVersionUrl/1.4.3-arm64 --- PASS: TestContainerdVersionUrl (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1- (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionUrl/-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionHash === RUN TestContainerdVersionHash/1.4.1- === RUN TestContainerdVersionHash/1.4.1-arm === RUN TestContainerdVersionHash/-amd64 === RUN TestContainerdVersionHash/-arm64 === RUN TestContainerdVersionHash/1.1.1-amd64 === RUN TestContainerdVersionHash/1.1.1-arm64 === RUN TestContainerdVersionHash/1.3.5-amd64 === RUN TestContainerdVersionHash/1.3.5-arm64 === RUN TestContainerdVersionHash/1.4.1-amd64 === RUN TestContainerdVersionHash/1.4.1-arm64 === RUN TestContainerdVersionHash/1.4.3-amd64 === RUN TestContainerdVersionHash/1.4.3-arm64 --- PASS: TestContainerdVersionHash (0.00s) --- PASS: TestContainerdVersionHash/1.4.1- (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionHash/-amd64 (0.00s) --- PASS: TestContainerdVersionHash/-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionsHashesAmd64 containerd_test.go:316: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesAmd64 (0.00s) === RUN TestContainerdVersionsHashesArm64 containerd_test.go:331: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesArm64 (0.00s) === RUN TestDeepValidate_OK --- PASS: TestDeepValidate_OK (0.00s) === RUN TestDeepValidate_NoNodeZones --- PASS: TestDeepValidate_NoNodeZones (0.00s) === RUN TestDeepValidate_NoMasterZones --- PASS: TestDeepValidate_NoMasterZones (0.00s) === RUN TestDeepValidate_BadZone deepvalidate_test.go:57: Zone validation not checked by DeepValidate --- SKIP: TestDeepValidate_BadZone (0.00s) === RUN TestDeepValidate_MixedRegion deepvalidate_test.go:69: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_MixedRegion (0.00s) === RUN TestDeepValidate_RegionAsZone deepvalidate_test.go:83: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_RegionAsZone (0.00s) === RUN TestDeepValidate_NotIncludedZone --- PASS: TestDeepValidate_NotIncludedZone (0.00s) === RUN TestDeepValidate_DuplicateZones --- PASS: TestDeepValidate_DuplicateZones (0.00s) === RUN TestDeepValidate_ExtraMasterZone --- PASS: TestDeepValidate_ExtraMasterZone (0.00s) === RUN TestDeepValidate_EvenEtcdClusterSize --- PASS: TestDeepValidate_EvenEtcdClusterSize (0.00s) === RUN TestDeepValidate_MissingEtcdMember --- PASS: TestDeepValidate_MissingEtcdMember (0.00s) === RUN TestPopulateClusterSpec_Proxy --- PASS: TestPopulateClusterSpec_Proxy (0.00s) === RUN TestPrecreateDNSNames --- PASS: TestPrecreateDNSNames (0.00s) === RUN TestDockerVersionUrlHash === RUN TestDockerVersionUrlHash/19.03.13-amd64 === RUN TestDockerVersionUrlHash/19.03.13-arm64 === RUN TestDockerVersionUrlHash/18.06.3-amd64 === RUN TestDockerVersionUrlHash/18.06.3-arm64 === RUN TestDockerVersionUrlHash/17.03.1-amd64 === RUN TestDockerVersionUrlHash/17.03.1-arm64 --- PASS: TestDockerVersionUrlHash (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-arm64 (0.00s) === RUN TestDockerVersionUrl === RUN TestDockerVersionUrl/19.03.13- === RUN TestDockerVersionUrl/19.03.13-arm === RUN TestDockerVersionUrl/-amd64 === RUN TestDockerVersionUrl/-arm64 === RUN TestDockerVersionUrl/18.06.3-amd64 === RUN TestDockerVersionUrl/18.06.3-arm64 === RUN TestDockerVersionUrl/19.03.13-amd64 === RUN TestDockerVersionUrl/19.03.13-arm64 --- PASS: TestDockerVersionUrl (0.00s) --- PASS: TestDockerVersionUrl/19.03.13- (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm (0.00s) --- PASS: TestDockerVersionUrl/-amd64 (0.00s) --- PASS: TestDockerVersionUrl/-arm64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm64 (0.00s) === RUN TestDockerVersionHash === RUN TestDockerVersionHash/19.03.13- === RUN TestDockerVersionHash/19.03.13-arm === RUN TestDockerVersionHash/-amd64 === RUN TestDockerVersionHash/-arm64 === RUN TestDockerVersionHash/1.1.1-amd64 === RUN TestDockerVersionHash/1.1.1-arm64 === RUN TestDockerVersionHash/19.03.13-amd64 === RUN TestDockerVersionHash/19.03.13-arm64 --- PASS: TestDockerVersionHash (0.00s) --- PASS: TestDockerVersionHash/19.03.13- (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm (0.00s) --- PASS: TestDockerVersionHash/-amd64 (0.00s) --- PASS: TestDockerVersionHash/-arm64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm64 (0.00s) === RUN TestDockerVersionsHashesAmd64 docker_test.go:245: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesAmd64 (0.00s) === RUN TestDockerVersionsHashesArm64 docker_test.go:260: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesArm64 (0.00s) === RUN Test_FindCNIAssetFromEnvironmentVariable --- PASS: Test_FindCNIAssetFromEnvironmentVariable (0.00s) === RUN Test_FindCNIAssetFromDefaults118 --- PASS: Test_FindCNIAssetFromDefaults118 (0.05s) === RUN Test_FindCNIAssetFromDefaults122 --- PASS: Test_FindCNIAssetFromDefaults122 (0.01s) === RUN TestRemoveSharedPrefix --- PASS: TestRemoveSharedPrefix (0.00s) === RUN TestCreateEtcdCluster --- PASS: TestCreateEtcdCluster (0.00s) === RUN TestSetupNetworking W0811 05:32:00.257842 40458 new_cluster.go:980] flannel UDP mode is not recommended; consider flannel-vxlan instead --- PASS: TestSetupNetworking (0.01s) === RUN TestPopulateCluster_Default_NoError --- PASS: TestPopulateCluster_Default_NoError (0.00s) === RUN TestPopulateCluster_Subnets === RUN TestPopulateCluster_Subnets/100.64.0.0/10 === RUN TestPopulateCluster_Subnets/10.0.0.0/9 === RUN TestPopulateCluster_Subnets/10.0.0.0/8 === RUN TestPopulateCluster_Subnets/::/0 --- PASS: TestPopulateCluster_Subnets (0.01s) --- PASS: TestPopulateCluster_Subnets/100.64.0.0/10 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/9 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/8 (0.00s) --- PASS: TestPopulateCluster_Subnets/::/0 (0.00s) === RUN TestPopulateCluster_Docker_Spec --- PASS: TestPopulateCluster_Docker_Spec (0.00s) === RUN TestPopulateCluster_StorageDefault --- PASS: TestPopulateCluster_StorageDefault (0.00s) === RUN TestPopulateCluster_Custom_CIDR --- PASS: TestPopulateCluster_Custom_CIDR (0.00s) === RUN TestPopulateCluster_IsolateMasters --- PASS: TestPopulateCluster_IsolateMasters (0.00s) === RUN TestPopulateCluster_IsolateMastersFalse --- PASS: TestPopulateCluster_IsolateMastersFalse (0.00s) === RUN TestPopulateCluster_Name_Required --- PASS: TestPopulateCluster_Name_Required (0.00s) === RUN TestPopulateCluster_Zone_Required --- PASS: TestPopulateCluster_Zone_Required (0.00s) === RUN TestPopulateCluster_NetworkCIDR_Required --- PASS: TestPopulateCluster_NetworkCIDR_Required (0.00s) === RUN TestPopulateCluster_NonMasqueradeCIDR_Required --- PASS: TestPopulateCluster_NonMasqueradeCIDR_Required (0.00s) === RUN TestPopulateCluster_CloudProvider_Required --- PASS: TestPopulateCluster_CloudProvider_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidNil_Required --- PASS: TestPopulateCluster_TopologyInvalidNil_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidValue_Required --- PASS: TestPopulateCluster_TopologyInvalidValue_Required (0.00s) === RUN TestPopulateCluster_BastionInvalidMatchingValues_Required --- PASS: TestPopulateCluster_BastionInvalidMatchingValues_Required (0.00s) === RUN TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required --- PASS: TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required (0.00s) === RUN TestPopulateCluster_APIServerCount --- PASS: TestPopulateCluster_APIServerCount (0.00s) === RUN TestPopulateCluster_AnonymousAuth --- PASS: TestPopulateCluster_AnonymousAuth (0.00s) === RUN TestPopulateCluster_DockerVersion --- PASS: TestPopulateCluster_DockerVersion (0.01s) === RUN TestPopulateCluster_KubeController_High_Enough_Version --- PASS: TestPopulateCluster_KubeController_High_Enough_Version (0.00s) === RUN TestPopulateInstanceGroup_Name_Required --- PASS: TestPopulateInstanceGroup_Name_Required (0.00s) === RUN TestPopulateInstanceGroup_Role_Required --- PASS: TestPopulateInstanceGroup_Role_Required (0.00s) === RUN TestPopulateInstanceGroup_Image_Required I0811 05:32:00.308365 40458 populate_instancegroup_spec.go:323] Cannot set default Image for CloudProvider="aws" --- PASS: TestPopulateInstanceGroup_Image_Required (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision --- PASS: TestPopulateInstanceGroup_AddTaintsCollision (0.00s) === RUN TestPopulateInstanceGroup_AddTaints --- PASS: TestPopulateInstanceGroup_AddTaints (0.00s) === RUN TestMachineArchitecture === RUN TestMachineArchitecture/t2.micro-amd64 === RUN TestMachineArchitecture/t3.micro-amd64 === RUN TestMachineArchitecture/a1.large-arm64 --- PASS: TestMachineArchitecture (0.00s) --- PASS: TestMachineArchitecture/t2.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/t3.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/a1.large-arm64 (0.00s) === RUN TestRuncVersionsHashesAmd64 runc_test.go:28: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesAmd64 (0.00s) === RUN TestRuncVersionsHashesArm64 runc_test.go:43: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesArm64 (0.00s) === RUN Test_AssignSubnets I0811 05:32:00.309537 40458 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0811 05:32:00.309577 40458 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0811 05:32:00.309595 40458 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I0811 05:32:00.309626 40458 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I0811 05:32:00.309645 40458 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet a I0811 05:32:00.309680 40458 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I0811 05:32:00.309715 40458 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet b I0811 05:32:00.309750 40458 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0811 05:32:00.309773 40458 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0811 05:32:00.309807 40458 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0811 05:32:00.309829 40458 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0811 05:32:00.309894 40458 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0811 05:32:00.309992 40458 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0811 05:32:00.310068 40458 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0811 05:32:00.310107 40458 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I0811 05:32:00.310153 40458 subnets.go:185] Assigned CIDR 10.96.0.0/11 to subnet b I0811 05:32:00.310197 40458 subnets.go:185] Assigned CIDR 10.128.0.0/11 to subnet c I0811 05:32:00.310239 40458 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet c I0811 05:32:00.310275 40458 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0811 05:32:00.310298 40458 subnets.go:199] Assigned CIDR 10.4.0.0/14 to subnet b I0811 05:32:00.310318 40458 subnets.go:199] Assigned CIDR 10.8.0.0/14 to subnet c --- PASS: Test_AssignSubnets (0.00s) === RUN Test_TemplateFunctions_CloudControllerConfigArgv === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller (0.00s) === RUN Test_KarpenterInstanceTypes I0811 05:32:00.312125 40458 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: Test_KarpenterInstanceTypes (0.00s) === RUN Test_BuildMirroredAsset === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-amd64 === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-arm64 === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup --- PASS: Test_BuildMirroredAsset (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-amd64 (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-arm64 (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup (0.00s) === RUN TestValidateFull_Default_Validates --- PASS: TestValidateFull_Default_Validates (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_NoDot --- PASS: TestValidateFull_ClusterName_InvalidDNS_NoDot (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_Invalid --- PASS: TestValidateFull_ClusterName_InvalidDNS_Invalid (0.00s) === RUN TestValidateFull_ClusterName_Required --- PASS: TestValidateFull_ClusterName_Required (0.00s) === RUN TestValidateFull_UpdatePolicy_Valid === RUN TestValidateFull_UpdatePolicy_Valid/missing === RUN TestValidateFull_UpdatePolicy_Valid/automatic === RUN TestValidateFull_UpdatePolicy_Valid/external --- PASS: TestValidateFull_UpdatePolicy_Valid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/missing (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/automatic (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/external (0.00s) === RUN TestValidateFull_UpdatePolicy_Invalid === RUN TestValidateFull_UpdatePolicy_Invalid/empty === RUN TestValidateFull_UpdatePolicy_Invalid/populated --- PASS: TestValidateFull_UpdatePolicy_Invalid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/empty (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/populated (0.00s) === RUN Test_Validate_Kubenet_With_14 --- PASS: Test_Validate_Kubenet_With_14 (0.00s) === RUN TestValidate_ClusterName_Import --- PASS: TestValidate_ClusterName_Import (0.00s) === RUN TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity --- PASS: TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup 1.424s === RUN TestGetASGTagsToDelete --- PASS: TestGetASGTagsToDelete (0.00s) === RUN TestProcessCompare --- PASS: TestProcessCompare (0.00s) === RUN TestAutoscalingGroupTerraformRender I0811 05:31:47.542044 38945 target.go:112] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender2607073196/001 I0811 05:31:47.544615 38945 target.go:112] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender2607073196/001 --- PASS: TestAutoscalingGroupTerraformRender (0.01s) === RUN TestAutoscalingGroupCloudformationRender I0811 05:31:47.546809 38945 target.go:168] Cloudformation output is in /tmp/TestAutoscalingGroupCloudformationRender1854601057/001 --- PASS: TestAutoscalingGroupCloudformationRender (0.00s) === RUN TestGetEBSVolumeTagsToDelete --- PASS: TestGetEBSVolumeTagsToDelete (0.00s) === RUN TestSharedEgressOnlyInternetGatewayDoesNotRename I0811 05:31:47.547423 38945 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0811 05:31:47.547502 38945 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0811 05:31:47.547559 38945 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { TagSpecifications: [{ ResourceType: "egress-only-internet-gateway", Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] }], VpcId: "vpc-1" } I0811 05:31:47.547729 38945 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0811 05:31:47.547806 38945 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:31:47.547864 38945 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0811 05:31:47.547882 38945 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0811 05:31:47.548072 38945 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0811 05:31:47.548107 38945 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0811 05:31:47.548145 38945 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0811 05:31:47.548215 38945 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0811 05:31:47.548291 38945 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0811 05:31:47.548419 38945 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0811 05:31:47.548472 38945 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:31:47.548512 38945 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0811 05:31:47.548527 38945 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0811 05:31:47.548557 38945 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0811 05:31:47.548604 38945 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0811 05:31:47.548660 38945 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedEgressOnlyInternetGatewayDoesNotRename (0.00s) === RUN TestElasticIPCreate I0811 05:31:47.548920 38945 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0811 05:31:47.548964 38945 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0811 05:31:47.549079 38945 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0811 05:31:47.549126 38945 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0811 05:31:47.549151 38945 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:31:47.549176 38945 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0811 05:31:47.549246 38945 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0811 05:31:47.549437 38945 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0811 05:31:47.549494 38945 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:31:47.549519 38945 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:31:47.549544 38945 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0811 05:31:47.549594 38945 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:31:47.549728 38945 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "eip1" }] }] } I0811 05:31:47.549791 38945 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:31:47.549813 38945 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedElasticIp", Value: "192.0.2.1" },{ Key: "AssociatedElasticIpAllocationId", Value: "eipalloc-1" }] } I0811 05:31:47.549857 38945 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0811 05:31:47.549986 38945 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0811 05:31:47.550032 38945 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0811 05:31:47.550074 38945 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0811 05:31:47.550089 38945 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0811 05:31:47.550113 38945 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:31:47.550136 38945 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0811 05:31:47.550185 38945 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0811 05:31:47.550252 38945 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:31:47.550276 38945 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0811 05:31:47.550312 38945 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:31:47.550356 38945 address.go:120] DescribeAddresses: { Filters: [{ Name: "public-ip", Values: ["192.0.2.1"] }] } I0811 05:31:47.550400 38945 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0811 05:31:47.550436 38945 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestElasticIPCreate (0.00s) === RUN TestSharedInternetGatewayDoesNotRename I0811 05:31:47.550573 38945 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0811 05:31:47.550614 38945 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0811 05:31:47.550636 38945 internetgateways.go:66] CreateInternetGateway: { } I0811 05:31:47.550659 38945 tags.go:42] CreateTags { Resources: ["igw-1"], Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] } I0811 05:31:47.550774 38945 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0811 05:31:47.550825 38945 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:31:47.550864 38945 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0811 05:31:47.550880 38945 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0811 05:31:47.550944 38945 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0811 05:31:47.550971 38945 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0811 05:31:47.551002 38945 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0811 05:31:47.551053 38945 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0811 05:31:47.551120 38945 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0811 05:31:47.551239 38945 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0811 05:31:47.551280 38945 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:31:47.551318 38945 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0811 05:31:47.551334 38945 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0811 05:31:47.551363 38945 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0811 05:31:47.551397 38945 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0811 05:31:47.551447 38945 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedInternetGatewayDoesNotRename (0.00s) === RUN TestLaunchTemplateCloudformationRender I0811 05:31:47.552230 38945 target.go:168] Cloudformation output is in /tmp/TestLaunchTemplateCloudformationRender2202758252/001 I0811 05:31:47.552494 38945 target.go:168] Cloudformation output is in /tmp/TestLaunchTemplateCloudformationRender2202758252/001 --- PASS: TestLaunchTemplateCloudformationRender (0.00s) === RUN TestLaunchTemplateTerraformRender I0811 05:31:47.554301 38945 target.go:112] Terraform output is in /tmp/TestLaunchTemplateTerraformRender3846218850/001 I0811 05:31:47.555800 38945 target.go:112] Terraform output is in /tmp/TestLaunchTemplateTerraformRender3846218850/001 --- PASS: TestLaunchTemplateTerraformRender (0.00s) === RUN TestParseRemovalRule --- PASS: TestParseRemovalRule (0.00s) === RUN TestPortRemovalRule --- PASS: TestPortRemovalRule (0.00s) === RUN TestPortRemovalRule_Zero --- PASS: TestPortRemovalRule_Zero (0.00s) === RUN TestSecurityGroupCreate I0811 05:31:47.556343 38945 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0811 05:31:47.556400 38945 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0811 05:31:47.556508 38945 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0811 05:31:47.556554 38945 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0811 05:31:47.556579 38945 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:31:47.556606 38945 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0811 05:31:47.556681 38945 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0811 05:31:47.556883 38945 securitygroups.go:40] CreateSecurityGroup: { Description: "Description", GroupName: "sg1", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "sg1" }] }], VpcId: "vpc-1" } I0811 05:31:47.556936 38945 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0811 05:31:47.556967 38945 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0811 05:31:47.557095 38945 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0811 05:31:47.557142 38945 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0811 05:31:47.557186 38945 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0811 05:31:47.557202 38945 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0811 05:31:47.557228 38945 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:31:47.557251 38945 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0811 05:31:47.557304 38945 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0811 05:31:47.557374 38945 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSecurityGroupCreate (0.00s) === RUN Test_Subnet_ValidateRequired --- PASS: Test_Subnet_ValidateRequired (0.00s) === RUN Test_Subnet_CannotChangeSubnet --- PASS: Test_Subnet_CannotChangeSubnet (0.00s) === RUN TestSubnetCreate I0811 05:31:47.557712 38945 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0811 05:31:47.557754 38945 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0811 05:31:47.557842 38945 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0811 05:31:47.557882 38945 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0811 05:31:47.557906 38945 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:31:47.557930 38945 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0811 05:31:47.557969 38945 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0811 05:31:47.558084 38945 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0811 05:31:47.558124 38945 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:31:47.558146 38945 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:31:47.558170 38945 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0811 05:31:47.558300 38945 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0811 05:31:47.558341 38945 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0811 05:31:47.558381 38945 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0811 05:31:47.558396 38945 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0811 05:31:47.558419 38945 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:31:47.558446 38945 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0811 05:31:47.558480 38945 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0811 05:31:47.558531 38945 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:31:47.558553 38945 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSubnetCreate (0.00s) === RUN TestSubnetCreateIPv6 I0811 05:31:47.558824 38945 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0811 05:31:47.558868 38945 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0811 05:31:47.558947 38945 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0811 05:31:47.558983 38945 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0811 05:31:47.559007 38945 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:31:47.559046 38945 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0811 05:31:47.559139 38945 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:31:47.559217 38945 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0811 05:31:47.559420 38945 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0811 05:31:47.559423 38945 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0811 05:31:47.559520 38945 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:31:47.559564 38945 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:31:47.559600 38945 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0811 05:31:47.559814 38945 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0811 05:31:47.559881 38945 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0811 05:31:47.559950 38945 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0811 05:31:47.559976 38945 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0811 05:31:47.560014 38945 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:31:47.560050 38945 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0811 05:31:47.560105 38945 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0811 05:31:47.560196 38945 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:31:47.560227 38945 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:31:47.560313 38945 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6 (0.00s) === RUN TestSubnetCreateIPv6NetNum I0811 05:31:47.560710 38945 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0811 05:31:47.560757 38945 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0811 05:31:47.560878 38945 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0811 05:31:47.560927 38945 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0811 05:31:47.560952 38945 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:31:47.560976 38945 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0811 05:31:47.561013 38945 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0811 05:31:47.561112 38945 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:31:47.561227 38945 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0811 05:31:47.561234 38945 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0811 05:31:47.561300 38945 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:31:47.561348 38945 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:31:47.561374 38945 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0811 05:31:47.561556 38945 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0811 05:31:47.561601 38945 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0811 05:31:47.561644 38945 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0811 05:31:47.561660 38945 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0811 05:31:47.561685 38945 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:31:47.561708 38945 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0811 05:31:47.561743 38945 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0811 05:31:47.561858 38945 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:31:47.561886 38945 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:31:47.561983 38945 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6NetNum (0.00s) === RUN TestSharedSubnetCreateDoesNotCreateNew I0811 05:31:47.562212 38945 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "ExistingVPC" }] }] } I0811 05:31:47.562316 38945 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "ExistingSubnet" }] }], VpcId: "vpc-1" } I0811 05:31:47.562529 38945 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0811 05:31:47.562594 38945 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:31:47.562655 38945 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0811 05:31:47.562680 38945 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0811 05:31:47.562782 38945 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0811 05:31:47.562823 38945 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0811 05:31:47.562877 38945 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0811 05:31:47.562925 38945 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:31:47.563071 38945 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0811 05:31:47.563109 38945 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0811 05:31:47.563151 38945 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0811 05:31:47.563351 38945 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0811 05:31:47.563409 38945 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:31:47.563464 38945 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0811 05:31:47.563487 38945 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0811 05:31:47.563530 38945 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0811 05:31:47.563577 38945 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0811 05:31:47.563653 38945 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedSubnetCreateDoesNotCreateNew (0.00s) === RUN TestVPCCreate I0811 05:31:47.563957 38945 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0811 05:31:47.564013 38945 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0811 05:31:47.564124 38945 vpcs.go:89] CreateVpc: { CidrBlock: "172.21.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0811 05:31:47.564180 38945 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0811 05:31:47.564212 38945 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:31:47.564242 38945 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0811 05:31:47.564372 38945 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0811 05:31:47.564430 38945 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0811 05:31:47.564499 38945 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0811 05:31:47.564523 38945 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0811 05:31:47.564558 38945 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0811 05:31:47.564591 38945 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestVPCCreate (0.00s) === RUN Test4758 --- PASS: Test4758 (0.00s) === RUN TestSharedVPCAdditionalCIDR I0811 05:31:47.564826 38945 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "172.22.0.0/16", VpcId: "vpc-1" } I0811 05:31:47.564991 38945 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0811 05:31:47.565048 38945 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc-1"] }] } I0811 05:31:47.565107 38945 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0811 05:31:47.565133 38945 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0811 05:31:47.565175 38945 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestSharedVPCAdditionalCIDR (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awstasks 0.076s === RUN TestValidateRegion --- PASS: TestValidateRegion (0.00s) === RUN TestFindRegion --- PASS: TestFindRegion (0.00s) === RUN TestEC2TagSpecification === RUN TestEC2TagSpecification/No_tags === RUN TestEC2TagSpecification/simple_tag --- PASS: TestEC2TagSpecification (0.00s) --- PASS: TestEC2TagSpecification/No_tags (0.00s) --- PASS: TestEC2TagSpecification/simple_tag (0.00s) === RUN Test_GetResourceName32 --- PASS: Test_GetResourceName32 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awsup 0.044s === RUN TestZoneToLocation === RUN TestZoneToLocation/test_case_0 === RUN TestZoneToLocation/test_case_1 === RUN TestZoneToLocation/test_case_2 --- PASS: TestZoneToLocation (0.00s) --- PASS: TestZoneToLocation/test_case_0 (0.00s) --- PASS: TestZoneToLocation/test_case_1 (0.00s) --- PASS: TestZoneToLocation/test_case_2 (0.00s) === RUN TestZoneToAvailabilityZoneNumber === RUN TestZoneToAvailabilityZoneNumber/test_case_0 === RUN TestZoneToAvailabilityZoneNumber/test_case_1 === RUN TestZoneToAvailabilityZoneNumber/test_case_2 --- PASS: TestZoneToAvailabilityZoneNumber (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_0 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_1 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_2 (0.00s) === RUN TestFindEtcdStatus --- PASS: TestFindEtcdStatus (0.00s) === RUN TestGetCloudGroups --- PASS: TestGetCloudGroups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azure 0.025s === RUN TestDiskRenderAzure I0811 05:31:35.804966 35784 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRenderAzure (0.00s) === RUN TestDiskFind --- PASS: TestDiskFind (0.00s) === RUN TestDiskRun I0811 05:31:35.805282 35784 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRun (0.00s) === RUN TestDiskCheckChanges === RUN TestDiskCheckChanges/test_case_0 === RUN TestDiskCheckChanges/test_case_1 === RUN TestDiskCheckChanges/test_case_2 === RUN TestDiskCheckChanges/test_case_3 --- PASS: TestDiskCheckChanges (0.00s) --- PASS: TestDiskCheckChanges/test_case_0 (0.00s) --- PASS: TestDiskCheckChanges/test_case_1 (0.00s) --- PASS: TestDiskCheckChanges/test_case_2 (0.00s) --- PASS: TestDiskCheckChanges/test_case_3 (0.00s) === RUN TestLoadBalancerRenderAzure I0811 05:31:35.805509 35784 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRenderAzure (0.00s) === RUN TestLoadBalancerFind loadbalancer_test.go:122: *azuretasks.LoadBalancer {"Name":"loadbalancer","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Subnet":{"Name":"subnet","Lifecycle":"","ResourceGroup":null,"VirtualNetwork":null,"CIDR":null,"Shared":null},"External":true,"Tags":null,"ForAPIServer":false} --- PASS: TestLoadBalancerFind (0.00s) === RUN TestLoadBalancerRun I0811 05:31:35.805931 35784 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRun (0.00s) === RUN TestLoadBalancerCheckChanges === RUN TestLoadBalancerCheckChanges/test_case_0 === RUN TestLoadBalancerCheckChanges/test_case_1 === RUN TestLoadBalancerCheckChanges/test_case_2 === RUN TestLoadBalancerCheckChanges/test_case_3 --- PASS: TestLoadBalancerCheckChanges (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_0 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_1 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_2 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_3 (0.00s) === RUN TestPublicIPAddressRenderAzure I0811 05:31:35.806132 35784 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRenderAzure (0.00s) === RUN TestPublicIPAddressFind publicipaddress_test.go:100: *azuretasks.PublicIPAddress {"Name":"publicIPAddress","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Tags":null} --- PASS: TestPublicIPAddressFind (0.00s) === RUN TestPublicIPAddressRun I0811 05:31:35.806393 35784 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRun (0.00s) === RUN TestPublicIPAddressCheckChanges === RUN TestPublicIPAddressCheckChanges/test_case_0 === RUN TestPublicIPAddressCheckChanges/test_case_1 === RUN TestPublicIPAddressCheckChanges/test_case_2 === RUN TestPublicIPAddressCheckChanges/test_case_3 --- PASS: TestPublicIPAddressCheckChanges (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_0 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_1 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_2 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_3 (0.00s) === RUN TestResourceGroupRenderAzure I0811 05:31:35.806604 35784 resourcegroup.go:103] Creating a new Resource Group with name: rg I0811 05:31:35.806625 35784 resourcegroup.go:105] Updating a Resource Group with name: rg --- PASS: TestResourceGroupRenderAzure (0.00s) === RUN TestResourceGroupFind --- PASS: TestResourceGroupFind (0.00s) === RUN TestResourceGroupRun I0811 05:31:35.806778 35784 resourcegroup.go:103] Creating a new Resource Group with name: rg --- PASS: TestResourceGroupRun (0.00s) === RUN TestResourceGroupCheckChanges === RUN TestResourceGroupCheckChanges/test_case_0 === RUN TestResourceGroupCheckChanges/test_case_1 === RUN TestResourceGroupCheckChanges/test_case_2 === RUN TestResourceGroupCheckChanges/test_case_3 --- PASS: TestResourceGroupCheckChanges (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_0 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_1 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_2 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_3 (0.00s) === RUN TestRoleAssignmentRenderAzure --- PASS: TestRoleAssignmentRenderAzure (0.00s) === RUN TestRoleAssignmentFind --- PASS: TestRoleAssignmentFind (0.00s) === RUN TestRoleAssignmentFind_NoPrincipalID --- PASS: TestRoleAssignmentFind_NoPrincipalID (0.00s) === RUN TestRoleAssignmentCheckChanges === RUN TestRoleAssignmentCheckChanges/test_case_0 === RUN TestRoleAssignmentCheckChanges/test_case_1 === RUN TestRoleAssignmentCheckChanges/test_case_2 === RUN TestRoleAssignmentCheckChanges/test_case_3 --- PASS: TestRoleAssignmentCheckChanges (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_0 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_1 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_2 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_3 (0.00s) === RUN TestSubnetRenderAzure I0811 05:31:35.807238 35784 subnet.go:106] Creating a new Subnet with name: vnet --- PASS: TestSubnetRenderAzure (0.00s) === RUN TestSubnetFind --- PASS: TestSubnetFind (0.00s) === RUN TestSubnetCheckChanges === RUN TestSubnetCheckChanges/test_case_0 === RUN TestSubnetCheckChanges/test_case_1 === RUN TestSubnetCheckChanges/test_case_2 === RUN TestSubnetCheckChanges/test_case_3 --- PASS: TestSubnetCheckChanges (0.00s) --- PASS: TestSubnetCheckChanges/test_case_0 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_1 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_2 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_3 (0.00s) === RUN TestVirtualNetworkRenderAzure I0811 05:31:35.807441 35784 virtualnetwork.go:116] Creating a new Virtual Network with name: vnet --- PASS: TestVirtualNetworkRenderAzure (0.00s) === RUN TestVirtualNetworkFind --- PASS: TestVirtualNetworkFind (0.00s) === RUN TestVirtualNetworkRun I0811 05:31:35.807655 35784 virtualnetwork.go:116] Creating a new Virtual Network with name: rg --- PASS: TestVirtualNetworkRun (0.00s) === RUN TestVirtualNetworkCheckChanges === RUN TestVirtualNetworkCheckChanges/test_case_0 === RUN TestVirtualNetworkCheckChanges/test_case_1 === RUN TestVirtualNetworkCheckChanges/test_case_2 === RUN TestVirtualNetworkCheckChanges/test_case_3 --- PASS: TestVirtualNetworkCheckChanges (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_0 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_1 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_2 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_3 (0.00s) === RUN TestSubnetIDParse --- PASS: TestSubnetIDParse (0.00s) === RUN TestLoadBalancerIDParse --- PASS: TestLoadBalancerIDParse (0.00s) === RUN TestVMScaleSetRenderAzure I0811 05:31:35.807923 35784 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRenderAzure (0.00s) === RUN TestVMScaleSetFind --- PASS: TestVMScaleSetFind (0.00s) === RUN TestVMScaleSetRun I0811 05:31:35.808173 35784 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRun (0.00s) === RUN TestVMScaleSetCheckChanges === RUN TestVMScaleSetCheckChanges/test_case_0 === RUN TestVMScaleSetCheckChanges/test_case_1 === RUN TestVMScaleSetCheckChanges/test_case_2 === RUN TestVMScaleSetCheckChanges/test_case_3 --- PASS: TestVMScaleSetCheckChanges (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_0 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_1 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_2 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_3 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azuretasks 0.029s ? k8s.io/kops/upup/pkg/fi/cloudup/bootstrapchannelbuilder [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/cloudformation [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/do [no test files] === RUN Test_Find === RUN Test_Find/successfully_found_volume === RUN Test_Find/no_volume_found === RUN Test_Find/error_from_server --- PASS: Test_Find (0.00s) --- PASS: Test_Find/successfully_found_volume (0.00s) --- PASS: Test_Find/no_volume_found (0.00s) --- PASS: Test_Find/error_from_server (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/dotasks 0.027s ? k8s.io/kops/upup/pkg/fi/cloudup/gce [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcediscovery [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcemetadata [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmsigner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmverifier [no test files] === RUN TestProjectIAMBinding I0811 05:31:45.117990 38660 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0811 05:31:45.118513 38660 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0811 05:31:45.118666 38660 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0811 05:31:45.119103 38660 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0811 05:31:45.119243 38660 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0811 05:31:45.119404 38660 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestProjectIAMBinding (0.00s) === RUN TestServiceAccount I0811 05:31:45.119637 38660 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0811 05:31:45.119739 38660 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0811 05:31:45.119869 38660 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0811 05:31:45.120021 38660 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0811 05:31:45.120163 38660 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0811 05:31:45.120225 38660 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestServiceAccount (0.00s) === RUN TestStorageBucketIAM I0811 05:31:45.120454 38660 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0811 05:31:45.120736 38660 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0811 05:31:45.120868 38660 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0811 05:31:45.121208 38660 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0811 05:31:45.121381 38660 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0811 05:31:45.121574 38660 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestStorageBucketIAM (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/gcetasks 0.084s ? k8s.io/kops/upup/pkg/fi/cloudup/hetzner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/hetznertasks [no test files] === RUN Test_OpenstackCloud_GetApiIngressStatus === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled --- PASS: Test_OpenstackCloud_GetApiIngressStatus (0.01s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstack 0.035s === RUN TestFilterPortsReturnsAllPorts --- PASS: TestFilterPortsReturnsAllPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPort --- PASS: TestFilterPortsReturnsOnlyTaggedPort (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPorts --- PASS: TestFilterPortsReturnsOnlyTaggedPorts (0.00s) === RUN Test_Port_GetDependencies --- PASS: Test_Port_GetDependencies (0.00s) === RUN Test_NewPortTaskFromCloud === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found === RUN Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups --- PASS: Test_NewPortTaskFromCloud (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found (0.00s) --- PASS: Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups (0.00s) === RUN Test_Port_Find === RUN Test_Port_Find/nothing_found === RUN Test_Port_Find/port_found_no_tags === RUN Test_Port_Find/port_found_with_tags === RUN Test_Port_Find/multiple_ports_found === RUN Test_Port_Find/error_listing_ports --- PASS: Test_Port_Find (0.00s) --- PASS: Test_Port_Find/nothing_found (0.00s) --- PASS: Test_Port_Find/port_found_no_tags (0.00s) --- PASS: Test_Port_Find/port_found_with_tags (0.00s) --- PASS: Test_Port_Find/multiple_ports_found (0.00s) --- PASS: Test_Port_Find/error_listing_ports (0.00s) === RUN Test_Port_CheckChanges === RUN Test_Port_CheckChanges/actual_nil_all_required_fields_set === RUN Test_Port_CheckChanges/actual_nil_required_field_Name_nil === RUN Test_Port_CheckChanges/actual_nil_required_field_Network_nil === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set --- PASS: Test_Port_CheckChanges (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_all_required_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Name_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Network_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set (0.00s) === RUN Test_Port_RenderOpenstack === RUN Test_Port_RenderOpenstack/actual_not_nil === RUN Test_Port_RenderOpenstack/actual_nil_success === RUN Test_Port_RenderOpenstack/actual_nil_cloud_error --- PASS: Test_Port_RenderOpenstack (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_not_nil (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_success (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_cloud_error (0.00s) === RUN Test_Port_createOptsFromPortTask === RUN Test_Port_createOptsFromPortTask/all_fields_set === RUN Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups --- PASS: Test_Port_createOptsFromPortTask (0.00s) --- PASS: Test_Port_createOptsFromPortTask/all_fields_set (0.00s) --- PASS: Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstacktasks 0.024s ? k8s.io/kops/upup/pkg/fi/cloudup/spotinsttasks [no test files] === RUN TestWriteValue === RUN TestWriteValue/null === RUN TestWriteValue/empty_list === RUN TestWriteValue/string === RUN TestWriteValue/list === RUN TestWriteValue/list_of_objects === RUN TestWriteValue/object_block --- PASS: TestWriteValue (0.00s) --- PASS: TestWriteValue/null (0.00s) --- PASS: TestWriteValue/empty_list (0.00s) --- PASS: TestWriteValue/string (0.00s) --- PASS: TestWriteValue/list (0.00s) --- PASS: TestWriteValue/list_of_objects (0.00s) --- PASS: TestWriteValue/object_block (0.00s) === RUN TestWriteLiteral === RUN TestWriteLiteral/string === RUN TestWriteLiteral/traversal === RUN TestWriteLiteral/provider_alias === RUN TestWriteLiteral/file --- PASS: TestWriteLiteral (0.00s) --- PASS: TestWriteLiteral/string (0.00s) --- PASS: TestWriteLiteral/traversal (0.00s) --- PASS: TestWriteLiteral/provider_alias (0.00s) --- PASS: TestWriteLiteral/file (0.00s) === RUN TestWriteLiteralList === RUN TestWriteLiteralList/empty_list === RUN TestWriteLiteralList/one_literal === RUN TestWriteLiteralList/two_literals === RUN TestWriteLiteralList/one_traversal_literal,_one_string_literal --- PASS: TestWriteLiteralList (0.00s) --- PASS: TestWriteLiteralList/empty_list (0.00s) --- PASS: TestWriteLiteralList/one_literal (0.00s) --- PASS: TestWriteLiteralList/two_literals (0.00s) --- PASS: TestWriteLiteralList/one_traversal_literal,_one_string_literal (0.00s) === RUN TestWriteMap === RUN TestWriteMap/empty_map === RUN TestWriteMap/simple_map === RUN TestWriteMap/complex_keys === RUN TestWriteMap/literal_list --- PASS: TestWriteMap (0.00s) --- PASS: TestWriteMap/empty_map (0.00s) --- PASS: TestWriteMap/simple_map (0.00s) --- PASS: TestWriteMap/complex_keys (0.00s) --- PASS: TestWriteMap/literal_list (0.00s) === RUN TestWriteMapLiterals === RUN TestWriteMapLiterals/literal_values --- PASS: TestWriteMapLiterals (0.00s) --- PASS: TestWriteMapLiterals/literal_values (0.00s) === RUN TestWriteLocalsOutputs === RUN TestWriteLocalsOutputs/empty_map === RUN TestWriteLocalsOutputs/single_output === RUN TestWriteLocalsOutputs/list_output --- PASS: TestWriteLocalsOutputs (0.00s) --- PASS: TestWriteLocalsOutputs/empty_map (0.00s) --- PASS: TestWriteLocalsOutputs/single_output (0.00s) --- PASS: TestWriteLocalsOutputs/list_output (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraform 0.028s === RUN TestGetOutputs === RUN TestGetOutputs/empty_map === RUN TestGetOutputs/single_output === RUN TestGetOutputs/list_output === RUN TestGetOutputs/duplicate_names === RUN TestGetOutputs/duplicate_values --- PASS: TestGetOutputs (0.00s) --- PASS: TestGetOutputs/empty_map (0.00s) --- PASS: TestGetOutputs/single_output (0.00s) --- PASS: TestGetOutputs/list_output (0.00s) --- PASS: TestGetOutputs/duplicate_names (0.00s) --- PASS: TestGetOutputs/duplicate_values (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraformWriter 0.032s === RUN TestKeypairDeps --- PASS: TestKeypairDeps (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/fitasks 0.025s ? k8s.io/kops/upup/pkg/fi/loader [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/cloudinit [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/local [no test files] === RUN TestArchiveDependencies --- PASS: TestArchiveDependencies (0.00s) === RUN TestBindMountCommands I0811 05:31:57.006180 40399 bindmount.go:224] running mount command [mount --bind /containerized_mounter /containerized_mounter] I0811 05:31:57.006270 40399 bindmount.go:233] running mount command [mount -o remount,exec /containerized_mounter] I0811 05:31:57.006284 40399 bindmount.go:224] running mount command [mount --rbind /var/lib/kubelet/ /containerized_mounter/rootfs/var/lib/kubelet] I0811 05:31:57.006296 40399 bindmount.go:244] running mount command [mount --make-rshared /containerized_mounter/rootfs/var/lib/kubelet] I0811 05:31:57.006307 40399 bindmount.go:224] running mount command [mount --bind -o ro /proc /containerized_mounter/rootfs/proc] I0811 05:31:57.006322 40399 bindmount.go:224] running mount command [mount --bind -o ro /dev /containerized_mounter/rootfs/dev] I0811 05:31:57.006341 40399 bindmount.go:224] running mount command [mount --bind -o ro /etc/resolv.conf /containerized_mounter/rootfs/etc/resolv.conf] --- PASS: TestBindMountCommands (0.00s) === RUN TestBindMountDependencies --- PASS: TestBindMountDependencies (0.00s) === RUN TestFileDependencies === RUN TestFileDependencies/user === RUN TestFileDependencies/parentDir === RUN TestFileDependencies/afterFiles --- PASS: TestFileDependencies (0.00s) --- PASS: TestFileDependencies/user (0.00s) --- PASS: TestFileDependencies/parentDir (0.00s) --- PASS: TestFileDependencies/afterFiles (0.00s) === RUN TestIssueCertFileDependencies --- PASS: TestIssueCertFileDependencies (0.00s) === RUN TestLoadImageTask_Deps --- PASS: TestLoadImageTask_Deps (0.00s) === RUN TestServiceTask_Deps --- PASS: TestServiceTask_Deps (0.00s) === RUN TestServiceTask_UnknownTypes W0811 05:31:57.007006 40399 service.go:93] Unhandled type *nodetasks.FakeTask in Service::GetDependencies: &{} --- PASS: TestServiceTask_UnknownTypes (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/nodeup/nodetasks 0.369s ? k8s.io/kops/upup/pkg/fi/nodeup/nodetasks/dnstasks [no test files] ? k8s.io/kops/upup/pkg/fi/secrets [no test files] === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) === RUN TestStringSlicesEqualIgnoreOrder --- PASS: TestStringSlicesEqualIgnoreOrder (0.00s) === RUN TestHashString --- PASS: TestHashString (0.00s) === RUN TestSanitizeString --- PASS: TestSanitizeString (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/utils 0.014s ? k8s.io/kops/upup/pkg/kutil [no test files] ? k8s.io/kops/upup/tools/generators/fitask [no test files] ? k8s.io/kops/util/pkg/architectures [no test files] === RUN TestFindDistribution --- PASS: TestFindDistribution (0.00s) PASS ok k8s.io/kops/util/pkg/distributions 0.004s ? k8s.io/kops/util/pkg/env [no test files] === RUN TestWithTee --- PASS: TestWithTee (0.00s) PASS ok k8s.io/kops/util/pkg/exec 0.003s === RUN TestNewHasher === RUN TestNewHasher/sha256 === RUN TestNewHasher/sha1 === RUN TestNewHasher/md5 --- PASS: TestNewHasher (0.00s) --- PASS: TestNewHasher/sha256 (0.00s) --- PASS: TestNewHasher/sha1 (0.00s) --- PASS: TestNewHasher/md5 (0.00s) === RUN TestMemberFromString === RUN TestMemberFromString/sha256_1 === RUN TestMemberFromString/sha256_2 === RUN TestMemberFromString/sha256_3 === RUN TestMemberFromString/sha1_1 === RUN TestMemberFromString/sha1_2 === RUN TestMemberFromString/sha1_3 === RUN TestMemberFromString/md5_1 === RUN TestMemberFromString/md5_2 === RUN TestMemberFromString/md5_3 --- PASS: TestMemberFromString (0.00s) --- PASS: TestMemberFromString/sha256_1 (0.00s) --- PASS: TestMemberFromString/sha256_2 (0.00s) --- PASS: TestMemberFromString/sha256_3 (0.00s) --- PASS: TestMemberFromString/sha1_1 (0.00s) --- PASS: TestMemberFromString/sha1_2 (0.00s) --- PASS: TestMemberFromString/sha1_3 (0.00s) --- PASS: TestMemberFromString/md5_1 (0.00s) --- PASS: TestMemberFromString/md5_2 (0.00s) --- PASS: TestMemberFromString/md5_3 (0.00s) === RUN TestFromString === RUN TestFromString/sha256 === RUN TestFromString/sha1 === RUN TestFromString/md5 === RUN TestFromString/unknown --- PASS: TestFromString (0.00s) --- PASS: TestFromString/sha256 (0.00s) --- PASS: TestFromString/sha1 (0.00s) --- PASS: TestFromString/md5 (0.00s) --- PASS: TestFromString/unknown (0.00s) PASS ok k8s.io/kops/util/pkg/hashing 0.005s === RUN TestKeys --- PASS: TestKeys (0.00s) === RUN TestSortedKeys --- PASS: TestSortedKeys (0.00s) PASS ok k8s.io/kops/util/pkg/maps 0.016s ? k8s.io/kops/util/pkg/mirrors [no test files] === RUN TestGetProxyEnvVars W0811 05:31:29.046677 33896 proxy.go:34] EgressProxy set but no proxy host provided --- PASS: TestGetProxyEnvVars (0.00s) PASS ok k8s.io/kops/util/pkg/proxy 0.100s === RUN TestSet === RUN TestSet/simple_setting === RUN TestSet/setting_with_wildcard === RUN TestSet/creating_missing_objects === RUN TestSet/set_int === RUN TestSet/set_int32 === RUN TestSet/set_int64 === RUN TestSet/set_int_pointer === RUN TestSet/set_int32_pointer === RUN TestSet/set_int64_pointer === RUN TestSet/set_enum === RUN TestSet/set_enum_slice === RUN TestSet/append_enum_slice --- PASS: TestSet (0.00s) --- PASS: TestSet/simple_setting (0.00s) --- PASS: TestSet/setting_with_wildcard (0.00s) --- PASS: TestSet/creating_missing_objects (0.00s) --- PASS: TestSet/set_int (0.00s) --- PASS: TestSet/set_int32 (0.00s) --- PASS: TestSet/set_int64 (0.00s) --- PASS: TestSet/set_int_pointer (0.00s) --- PASS: TestSet/set_int32_pointer (0.00s) --- PASS: TestSet/set_int64_pointer (0.00s) --- PASS: TestSet/set_enum (0.00s) --- PASS: TestSet/set_enum_slice (0.00s) --- PASS: TestSet/append_enum_slice (0.00s) === RUN TestSetInvalidPath === RUN TestSetInvalidPath/setting_with_wildcard === RUN TestSetInvalidPath/creating_missing_objects === RUN TestSetInvalidPath/set_int === RUN TestSetInvalidPath/set_int32 === RUN TestSetInvalidPath/set_int64 --- PASS: TestSetInvalidPath (0.00s) --- PASS: TestSetInvalidPath/setting_with_wildcard (0.00s) --- PASS: TestSetInvalidPath/creating_missing_objects (0.00s) --- PASS: TestSetInvalidPath/set_int (0.00s) --- PASS: TestSetInvalidPath/set_int32 (0.00s) --- PASS: TestSetInvalidPath/set_int64 (0.00s) === RUN TestUnset === RUN TestUnset/simple_unsetting === RUN TestUnset/unsetting_with_wildcard === RUN TestUnset/uset_int === RUN TestUnset/unset_int32 === RUN TestUnset/unset_int64 === RUN TestUnset/unset_int_pointer === RUN TestUnset/unset_int32_pointer === RUN TestUnset/unset_int64_pointer === RUN TestUnset/unset_enum === RUN TestUnset/unset_enum_slice --- PASS: TestUnset (0.00s) --- PASS: TestUnset/simple_unsetting (0.00s) --- PASS: TestUnset/unsetting_with_wildcard (0.00s) --- PASS: TestUnset/uset_int (0.00s) --- PASS: TestUnset/unset_int32 (0.00s) --- PASS: TestUnset/unset_int64 (0.00s) --- PASS: TestUnset/unset_int_pointer (0.00s) --- PASS: TestUnset/unset_int32_pointer (0.00s) --- PASS: TestUnset/unset_int64_pointer (0.00s) --- PASS: TestUnset/unset_enum (0.00s) --- PASS: TestUnset/unset_enum_slice (0.00s) === RUN TestUnsetInvalidPath === RUN TestUnsetInvalidPath/usetting_with_wildcard === RUN TestUnsetInvalidPath/missing_objects --- PASS: TestUnsetInvalidPath (0.00s) --- PASS: TestUnsetInvalidPath/usetting_with_wildcard (0.00s) --- PASS: TestUnsetInvalidPath/missing_objects (0.00s) === RUN TestParseFieldPath === RUN TestParseFieldPath/test_Spec.Containers.Image === RUN TestParseFieldPath/test_Spec.Containers[0].Image === RUN TestParseFieldPath/test_Spec.Containers[*].Image --- PASS: TestParseFieldPath (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers.Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[0].Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[*].Image (0.00s) PASS ok k8s.io/kops/util/pkg/reflectutils 0.009s === RUN TestGetUniqueStrings --- PASS: TestGetUniqueStrings (0.00s) === RUN TestContains --- PASS: TestContains (0.00s) PASS ok k8s.io/kops/util/pkg/slice 0.006s ? k8s.io/kops/util/pkg/tables [no test files] === RUN TestSplitToSections --- PASS: TestSplitToSections (0.00s) PASS ok k8s.io/kops/util/pkg/text 0.003s ? k8s.io/kops/util/pkg/ui [no test files] === RUN TestAzureBlobPathBase === RUN TestAzureBlobPathBase/Test_case_0 === RUN TestAzureBlobPathBase/Test_case_1 === RUN TestAzureBlobPathBase/Test_case_2 --- PASS: TestAzureBlobPathBase (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_2 (0.00s) === RUN TestAzureBlobPathPath === RUN TestAzureBlobPathPath/Test_case_0 === RUN TestAzureBlobPathPath/Test_case_1 === RUN TestAzureBlobPathPath/Test_case_2 --- PASS: TestAzureBlobPathPath (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_2 (0.00s) === RUN TestAzureBlobPathJoin --- PASS: TestAzureBlobPathJoin (0.00s) === RUN TestBuildVaultPath context_test.go:27: No vault dev token set --- SKIP: TestBuildVaultPath (0.00s) === RUN TestCreateFile --- PASS: TestCreateFile (0.00s) === RUN TestWriteTo --- PASS: TestWriteTo (0.00s) === RUN TestMemFsCreateFile --- PASS: TestMemFsCreateFile (0.00s) === RUN TestMemFsReadDir --- PASS: TestMemFsReadDir (0.00s) === RUN TestMemFsReadTree --- PASS: TestMemFsReadTree (0.00s) === RUN Test_VFSPath --- PASS: Test_VFSPath (0.00s) === RUN Test_S3Path_Parse --- PASS: Test_S3Path_Parse (0.00s) === RUN Test_newVaultPath vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_newVaultPath (0.00s) === RUN Test_newVaultPathHostOnly --- PASS: Test_newVaultPathHostOnly (0.00s) === RUN Test_WriteFileReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileReadFile (0.00s) === RUN Test_WriteFileDeleteFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileDeleteFile (0.00s) === RUN Test_CreateFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_CreateFile (0.00s) === RUN Test_ReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadFile (0.00s) === RUN Test_Join vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_Join (0.00s) === RUN Test_VaultReadDirList vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadDirList (0.00s) === RUN Test_ReadDir vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadDir (0.00s) === RUN Test_IsDirectory vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_IsDirectory (0.00s) === RUN Test_VaultReadTree vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadTree (0.00s) PASS ok k8s.io/kops/util/pkg/vfs 0.079s === RUN TestGSRenderTerraform === RUN TestGSRenderTerraform/gs://foo/bar --- PASS: TestGSRenderTerraform (0.00s) --- PASS: TestGSRenderTerraform/gs://foo/bar (0.00s) === RUN TestS3RenderTerraform === RUN TestS3RenderTerraform/s3://foo/bar I0811 05:31:46.077870 38815 s3context.go:90] Found S3_ENDPOINT="foo.s3.amazonaws.com", using as non-AWS S3 backend W0811 05:31:46.077924 38815 s3context.go:238] Unable to read bucket encryption policy for "foo" in region "us-east-1": will encrypt using AES256 --- PASS: TestS3RenderTerraform (0.00s) --- PASS: TestS3RenderTerraform/s3://foo/bar (0.00s) PASS ok k8s.io/kops/util/pkg/vfs/tests 0.059s >>> kops: Entering fakeroot... >>> kops*: Running postcheck for kops >>> kops*: Preparing package kops... >>> kops*: Stripping binaries >>> kops*: Scanning shared objects >>> kops*: Tracing dependencies... >>> kops*: Package size: 123.5 MB >>> kops*: Compressing data... >>> kops*: Create checksum... >>> kops*: Create kops-1.24.1-r0.apk >>> kops: Build complete at Thu, 11 Aug 2022 05:32:48 +0000 elapsed time 0h 3m 58s >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Uninstalling dependencies... (1/2) Purging .makedepends-kops (20220811.052852) (2/2) Purging kubernetes (1.24.3-r2) OK: 717 MiB in 116 packages >>> kops: Updating the testing/x86_64 repository index... >>> kops: Signing the index...