>>> kops: Building testing/kops 1.25.0-r0 (using abuild 3.10.0_rc1-r1) started Tue, 20 Sep 2022 05:46:37 +0000 >>> kops: Checking sanity of /home/buildozer/aports/testing/kops/APKBUILD... >>> kops: Analyzing dependencies... >>> kops: Installing for build: build-base kubernetes go (1/3) Installing kubernetes (1.25.0-r1) (2/3) Installing go (1.19.1-r0) (3/3) Installing .makedepends-kops (20220920.054639) Executing busybox-1.35.0-r25.trigger OK: 806 MiB in 101 packages >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.25.0.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 146 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 curl: (22) The requested URL returned error: 404 >>> kops: Fetching kops-1.25.0.tar.gz::https://github.com/kubernetes/kops/archive/v1.25.0.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 2624k 0 2624k 0 0 3039k 0 --:--:-- --:--:-- --:--:-- 3039k 100 11.2M 0 11.2M 0 0 6212k 0 --:--:-- 0:00:01 --:--:-- 8952k 100 16.0M 0 16.0M 0 0 5711k 0 --:--:-- 0:00:02 --:--:-- 6860k 100 21.4M 0 21.4M 0 0 5676k 0 --:--:-- 0:00:03 --:--:-- 6437k 100 25.3M 0 25.3M 0 0 5341k 0 --:--:-- 0:00:04 --:--:-- 5837k 100 30.1M 0 30.1M 0 0 5262k 0 --:--:-- 0:00:05 --:--:-- 5646k 100 30.7M 0 30.7M 0 0 5211k 0 --:--:-- 0:00:06 --:--:-- 4766k >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.25.0.tar.gz >>> kops: Checking sha512sums... kops-1.25.0.tar.gz: OK >>> kops: Unpacking /var/cache/distfiles/edge/kops-1.25.0.tar.gz... go: downloading k8s.io/klog/v2 v2.70.1 go: downloading github.com/blang/semver/v4 v4.0.0 go: downloading github.com/aws/aws-sdk-go v1.44.85 go: downloading github.com/cert-manager/cert-manager v1.9.1 go: downloading go.uber.org/multierr v1.8.0 go: downloading k8s.io/api v0.25.0 go: downloading k8s.io/apimachinery v0.25.0 go: downloading k8s.io/client-go v0.25.0 go: downloading google.golang.org/api v0.94.0 go: downloading k8s.io/klog v1.0.0 go: downloading github.com/gophercloud/gophercloud v0.25.0 go: downloading github.com/google/uuid v1.3.0 go: downloading github.com/spf13/cobra v1.5.0 go: downloading github.com/aws/amazon-ec2-instance-selector/v2 v2.4.0 go: downloading github.com/spf13/viper v1.12.0 go: downloading github.com/google/go-containerregistry v0.11.0 go: downloading k8s.io/cli-runtime v0.25.0 go: downloading github.com/spf13/pflag v1.0.5 go: downloading golang.org/x/crypto v0.0.0-20220824171710-5757bc0c5503 go: downloading helm.sh/helm/v3 v3.9.4 go: downloading github.com/go-logr/logr v1.2.3 go: downloading sigs.k8s.io/controller-runtime v0.12.3 go: downloading sigs.k8s.io/yaml v1.3.0 go: downloading k8s.io/kubectl v0.25.0 go: downloading golang.org/x/oauth2 v0.0.0-20220822191816-0ebed06d0094 go: downloading github.com/stretchr/testify v1.8.0 go: downloading cloud.google.com/go/compute v1.9.0 go: downloading gopkg.in/gcfg.v1 v1.2.3 go: downloading k8s.io/legacy-cloud-providers v0.25.0 go: downloading gopkg.in/inf.v0 v0.9.1 go: downloading go.uber.org/atomic v1.9.0 go: downloading github.com/gogo/protobuf v1.3.2 go: downloading github.com/google/gofuzz v1.2.0 go: downloading github.com/inconshreveable/mousetrap v1.0.0 go: downloading github.com/apparentlymart/go-cidr v1.1.0 go: downloading github.com/fsnotify/fsnotify v1.5.4 go: downloading github.com/mitchellh/mapstructure v1.5.0 go: downloading github.com/spf13/afero v1.8.2 go: downloading github.com/spf13/cast v1.5.0 go: downloading github.com/spf13/jwalterweatherman v1.1.0 go: downloading github.com/Azure/azure-pipeline-go v0.2.3 go: downloading github.com/Azure/azure-storage-blob-go v0.15.0 go: downloading github.com/go-ini/ini v1.67.0 go: downloading github.com/hashicorp/vault/api v1.7.2 go: downloading github.com/pkg/sftp v1.13.5 go: downloading github.com/mitchellh/go-homedir v1.1.0 go: downloading k8s.io/component-base v0.25.0 go: downloading github.com/prometheus/client_golang v1.13.0 go: downloading github.com/imdario/mergo v0.3.12 go: downloading github.com/davecgh/go-spew v1.1.1 go: downloading github.com/google/gnostic v0.6.9 go: downloading github.com/google/go-cmp v0.5.8 go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.2 go: downloading gopkg.in/yaml.v2 v2.4.0 go: downloading golang.org/x/net v0.0.0-20220822230855-b0a4917ee28c go: downloading k8s.io/utils v0.0.0-20220823124924-e9cbc92d1a73 go: downloading golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 go: downloading github.com/digitalocean/godo v1.83.0 go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.2.3 go: downloading github.com/evanphx/json-patch v5.6.0+incompatible go: downloading github.com/opencontainers/go-digest v1.0.0 go: downloading golang.org/x/sync v0.0.0-20220819030929-7fc1605a5dde go: downloading github.com/Azure/azure-sdk-for-go v66.0.0+incompatible go: downloading github.com/Azure/go-autorest/autorest/azure/auth v0.5.11 go: downloading github.com/MakeNowJust/heredoc/v2 v2.0.1 go: downloading k8s.io/cloud-provider-aws v1.24.1 go: downloading github.com/hetznercloud/hcloud-go v1.35.2 go: downloading github.com/pelletier/go-toml v1.9.5 go: downloading k8s.io/kubelet v0.25.0 go: downloading github.com/MakeNowJust/heredoc v1.0.0 go: downloading github.com/Azure/go-autorest/autorest v0.11.28 go: downloading k8s.io/mount-utils v0.25.0 go: downloading github.com/pkg/errors v0.9.1 go: downloading golang.org/x/sys v0.0.0-20220825204002-c680a09ffe64 go: downloading github.com/Masterminds/sprig/v3 v3.2.2 go: downloading github.com/Azure/go-autorest v14.2.0+incompatible go: downloading github.com/sergi/go-diff v1.2.0 go: downloading gopkg.in/square/go-jose.v2 v2.6.0 go: downloading github.com/Azure/go-autorest/autorest/to v0.4.0 go: downloading google.golang.org/appengine v1.6.7 go: downloading github.com/chai2010/gettext-go v1.0.2 go: downloading github.com/russross/blackfriday v1.6.0 go: downloading github.com/mitchellh/go-wordwrap v1.0.0 go: downloading github.com/spotinst/spotinst-sdk-go v1.126.0 go: downloading github.com/jacksontj/memberlistmesh v0.0.0-20190905163944-93462b9d2bb7 go: downloading github.com/weaveworks/mesh v0.0.0-20191105120815-58dbcc3e8e63 go: downloading github.com/pmezard/go-difflib v1.0.0 go: downloading gopkg.in/yaml.v3 v3.0.1 go: downloading github.com/google/go-tpm-tools v0.3.9 go: downloading github.com/google/go-tpm v0.3.3 go: downloading github.com/hashicorp/hcl/v2 v2.13.0 go: downloading github.com/zclconf/go-cty v1.11.0 go: downloading k8s.io/gengo v0.0.0-20220613173612-397b4ae3bce7 go: downloading gopkg.in/warnings.v0 v0.1.2 go: downloading github.com/GoogleCloudPlatform/k8s-cloud-provider v1.18.1-0.20220218231025-f11817397a1b go: downloading k8s.io/cloud-provider v0.25.0 go: downloading sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2 go: downloading k8s.io/apiextensions-apiserver v0.24.2 go: downloading sigs.k8s.io/gateway-api v0.4.3 go: downloading github.com/golang/protobuf v1.5.2 go: downloading google.golang.org/protobuf v1.28.1 go: downloading golang.org/x/time v0.0.0-20220210224613-90d013bbcef8 go: downloading golang.org/x/text v0.3.7 go: downloading github.com/subosito/gotenv v1.3.0 go: downloading github.com/hashicorp/hcl v1.0.0 go: downloading gopkg.in/ini.v1 v1.66.6 go: downloading github.com/magiconair/properties v1.8.6 go: downloading github.com/pelletier/go-toml/v2 v2.0.1 go: downloading github.com/mattn/go-ieproxy v0.0.1 go: downloading github.com/Azure/go-autorest/autorest/adal v0.9.20 go: downloading github.com/patrickmn/go-cache v2.1.0+incompatible go: downloading github.com/charmbracelet/bubbles v0.13.0 go: downloading github.com/charmbracelet/bubbletea v0.21.0 go: downloading github.com/charmbracelet/lipgloss v0.5.0 go: downloading github.com/evertras/bubble-table v0.14.4 go: downloading github.com/muesli/termenv v0.11.1-0.20220212125758-44cd13922739 go: downloading k8s.io/kube-openapi v0.0.0-20220803162953-67bda5d908f1 go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de go: downloading sigs.k8s.io/kustomize/api v0.12.1 go: downloading sigs.k8s.io/kustomize/kyaml v0.13.9 go: downloading github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 go: downloading github.com/peterbourgon/diskv v2.0.1+incompatible go: downloading github.com/docker/cli v20.10.17+incompatible go: downloading github.com/docker/distribution v2.8.1+incompatible go: downloading github.com/containerd/stargz-snapshotter/estargz v0.12.0 go: downloading github.com/russross/blackfriday/v2 v2.1.0 go: downloading github.com/cenkalti/backoff/v3 v3.0.0 go: downloading github.com/hashicorp/errwrap v1.1.0 go: downloading github.com/hashicorp/go-cleanhttp v0.5.2 go: downloading github.com/hashicorp/go-multierror v1.1.1 go: downloading github.com/hashicorp/go-retryablehttp v0.6.6 go: downloading github.com/hashicorp/go-rootcerts v1.0.2 go: downloading github.com/hashicorp/go-secure-stdlib/parseutil v0.1.6 go: downloading github.com/hashicorp/vault/sdk v0.5.1 go: downloading github.com/hashicorp/go-hclog v1.2.0 go: downloading github.com/kr/fs v0.1.0 go: downloading github.com/moby/sys/mountinfo v0.6.0 go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da go: downloading gomodules.xyz/jsonpatch/v2 v2.2.0 go: downloading github.com/google/go-querystring v1.1.0 go: downloading github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6 go: downloading github.com/Masterminds/goutils v1.1.1 go: downloading github.com/Masterminds/semver/v3 v3.1.1 go: downloading github.com/huandu/xstrings v1.3.2 go: downloading github.com/mitchellh/copystructure v1.2.0 go: downloading github.com/shopspring/decimal v1.2.0 go: downloading github.com/prometheus/client_model v0.2.0 go: downloading github.com/prometheus/common v0.37.0 go: downloading github.com/beorn7/perks v1.0.1 go: downloading github.com/cespare/xxhash/v2 v2.1.2 go: downloading github.com/prometheus/procfs v0.8.0 go: downloading github.com/Azure/go-autorest/autorest/azure/cli v0.4.5 go: downloading github.com/Azure/go-autorest/logger v0.2.1 go: downloading github.com/dimchansky/utfbom v1.1.1 go: downloading github.com/json-iterator/go v1.1.12 go: downloading k8s.io/csi-translation-lib v0.25.0 go: downloading github.com/hashicorp/go-sockaddr v1.0.2 go: downloading github.com/hashicorp/memberlist v0.3.1 go: downloading github.com/oklog/ulid v1.3.1 go: downloading github.com/Azure/go-autorest/tracing v0.6.0 go: downloading golang.org/x/tools v0.1.8 go: downloading github.com/googleapis/gax-go/v2 v2.4.0 go: downloading google.golang.org/grpc v1.48.0 go: downloading github.com/jmespath/go-jmespath v0.4.0 go: downloading github.com/oliveagle/jsonpath v0.0.0-20180606110733-2e52cf6e6852 go: downloading github.com/agext/levenshtein v1.2.1 go: downloading github.com/apparentlymart/go-textseg/v13 v13.0.0 go: downloading github.com/Azure/go-autorest/autorest/date v0.3.0 go: downloading github.com/golang-jwt/jwt/v4 v4.2.0 go: downloading github.com/containerd/console v1.0.3 go: downloading github.com/mattn/go-isatty v0.0.14 go: downloading github.com/muesli/ansi v0.0.0-20211018074035-2e021307bc4b go: downloading github.com/muesli/cancelreader v0.2.0 go: downloading github.com/onsi/ginkgo v1.16.5 go: downloading github.com/onsi/gomega v1.19.0 go: downloading go.uber.org/goleak v1.1.12 go: downloading github.com/frankban/quicktest v1.14.3 go: downloading gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c go: downloading github.com/go-logr/zapr v1.2.3 go: downloading github.com/go-test/deep v1.0.3 go: downloading go.uber.org/zap v1.19.1 go: downloading github.com/Azure/go-autorest/autorest/mocks v0.4.2 go: downloading github.com/kylelemons/godebug v1.1.0 go: downloading github.com/kr/pretty v0.3.0 go: downloading github.com/muesli/reflow v0.3.0 go: downloading github.com/lucasb-eyer/go-colorful v1.2.0 go: downloading github.com/mattn/go-runewidth v0.0.13 go: downloading github.com/atotto/clipboard v0.1.4 go: downloading github.com/sahilm/fuzzy v0.1.0 go: downloading github.com/opencontainers/image-spec v1.0.3-0.20220114050600-8b9d41f48198 go: downloading github.com/google/btree v1.0.1 go: downloading github.com/vbatts/tar-split v0.11.2 go: downloading github.com/klauspost/compress v1.15.8 go: downloading github.com/google/go-attestation v0.4.4-0.20220404204839-8820d49b18d9 go: downloading github.com/hashicorp/go-secure-stdlib/strutil v0.1.2 go: downloading github.com/fatih/color v1.13.0 go: downloading github.com/mattn/go-colorable v0.1.12 go: downloading github.com/containerd/containerd v1.6.6 go: downloading github.com/sirupsen/logrus v1.9.0 go: downloading oras.land/oras-go v1.2.0 go: downloading github.com/distribution/distribution/v3 v3.0.0-20220526142353-ffbd94cbe269 go: downloading github.com/phayes/freeport v0.0.0-20220201140144-74d24b5ae9f5 go: downloading github.com/mitchellh/go-testing-interface v1.0.0 go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 go: downloading github.com/go-openapi/swag v0.19.14 go: downloading github.com/onsi/ginkgo/v2 v2.1.4 go: downloading github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d go: downloading github.com/creack/pty v1.1.11 go: downloading gotest.tools/v3 v3.0.3 go: downloading github.com/mitchellh/reflectwalk v1.0.2 go: downloading github.com/docker/docker v20.10.17+incompatible go: downloading github.com/stretchr/objx v0.4.0 go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 go: downloading golang.org/x/xerrors v0.0.0-20220609144429-65e65417b02f go: downloading go.opencensus.io v0.23.0 go: downloading github.com/armon/go-metrics v0.3.10 go: downloading github.com/hashicorp/go-msgpack v0.5.3 go: downloading github.com/miekg/dns v1.1.48 go: downloading github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd go: downloading github.com/modern-go/reflect2 v1.0.2 go: downloading google.golang.org/genproto v0.0.0-20220804142021-4e6b2dfa6612 go: downloading github.com/kr/text v0.2.0 go: downloading github.com/rogpeppe/go-internal v1.8.0 go: downloading github.com/rivo/uniseg v0.2.0 go: downloading github.com/ryanuber/go-glob v1.0.0 go: downloading github.com/jmespath/go-jmespath/internal/testify v1.5.1 go: downloading github.com/google/go-tspi v0.2.1-0.20190423175329-115dea689aad go: downloading github.com/docker/go-connections v0.4.0 go: downloading github.com/golang/snappy v0.0.4 go: downloading github.com/pierrec/lz4 v2.5.2+incompatible go: downloading github.com/hashicorp/golang-lru v0.5.4 go: downloading github.com/hashicorp/go-plugin v1.4.3 go: downloading github.com/armon/go-radix v1.0.0 go: downloading github.com/hashicorp/go-secure-stdlib/mlock v0.1.1 go: downloading github.com/hashicorp/go-uuid v1.0.2 go: downloading github.com/hashicorp/go-version v1.2.1 go: downloading github.com/emicklei/go-restful/v3 v3.8.0 go: downloading github.com/go-errors/errors v1.0.1 go: downloading github.com/go-openapi/jsonreference v0.19.5 go: downloading github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 go: downloading github.com/moby/spdystream v0.2.0 go: downloading github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 go: downloading github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153 go: downloading github.com/docker/docker-credential-helpers v0.6.4 go: downloading github.com/googleapis/enterprise-certificate-proxy v0.1.0 go: downloading github.com/mailru/easyjson v0.7.6 go: downloading github.com/hashicorp/go-immutable-radix v1.3.1 go: downloading github.com/pascaldekloe/goe v0.1.0 go: downloading github.com/google/certificate-transparency-go v1.1.2 go: downloading github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d go: downloading github.com/bshuster-repo/logrus-logstash-hook v1.0.0 go: downloading github.com/bugsnag/bugsnag-go v0.0.0-20141110184014-b1d153021fcd go: downloading github.com/docker/go-metrics v0.0.1 go: downloading github.com/docker/libtrust v0.0.0-20150114040149-fa567046d9b1 go: downloading github.com/gorilla/handlers v1.5.1 go: downloading github.com/yvasiyarov/gorelic v0.0.0-20141212073537-a9bba5b9ab50 go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 go: downloading github.com/xlab/treeprint v1.1.0 go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: downloading github.com/hashicorp/yamux v0.0.0-20180604194846-3520598351bb go: downloading github.com/oklog/run v1.0.0 go: downloading github.com/jhump/protoreflect v1.6.1 go: downloading github.com/fatih/structs v1.1.0 go: downloading github.com/PuerkitoBio/purell v1.1.1 go: downloading github.com/go-openapi/jsonpointer v0.19.5 go: downloading github.com/nxadm/tail v1.4.8 go: downloading github.com/josharian/intern v1.0.0 go: downloading github.com/gorilla/mux v1.8.0 go: downloading github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c go: downloading github.com/gomodule/redigo v1.8.2 go: downloading github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0 go: downloading github.com/felixge/httpsnoop v1.0.1 go: downloading github.com/yvasiyarov/go-metrics v0.0.0-20140926110328-57bccd1ccd43 go: downloading github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f go: downloading github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 go: downloading gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 go: downloading github.com/bugsnag/osext v0.0.0-20130617224835-0dd3f918b21b go: downloading github.com/moby/locker v1.0.1 go: downloading go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 go: downloading github.com/Microsoft/hcsshim v0.9.3 go: downloading golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4 go: downloading github.com/Microsoft/go-winio v0.5.2 go: downloading github.com/containerd/cgroups v1.0.3 go: downloading github.com/docker/go-units v0.4.0 go: downloading github.com/morikuni/aec v1.0.0 go: downloading github.com/Azure/go-autorest/autorest/validation v0.3.1 fatal: not a git repository (or any of the parent directories): .git /bin/sh: tools/get_version.sh: not found mkdir -p /home/buildozer/aports/testing/kops/src/kops-1.25.0/.build/dist/linux/amd64 GOOS=linux GOARCH=amd64 go build "-trimpath" -o /home/buildozer/aports/testing/kops/src/kops-1.25.0/.build/dist/linux/amd64/kops -ldflags=all=" -X k8s.io/kops.Version= -X k8s.io/kops.GitVersion=" k8s.io/kops/cmd/kops fatal: not a git repository (or any of the parent directories): .git go test -v ./... ? k8s.io/kops [no test files] ? k8s.io/kops/channels/cmd/channels [no test files] ? k8s.io/kops/channels/pkg/api [no test files] === RUN Test_Filtering --- PASS: Test_Filtering (0.00s) === RUN Test_Replacement --- PASS: Test_Replacement (0.00s) === RUN Test_GetRequiredUpdates --- PASS: Test_GetRequiredUpdates (0.00s) === RUN Test_NeedsRollingUpdate I0920 05:48:47.688908 5070 addon.go:233] addon test wants to update all nodes I0920 05:48:47.690972 5070 addon.go:233] addon test wants to update worker nodes I0920 05:48:47.691356 5070 addon.go:233] addon test wants to update control-plane nodes --- PASS: Test_NeedsRollingUpdate (0.00s) === RUN Test_InstallPKI I0920 05:48:47.691776 5070 addon.go:266] installing PKI for "test" I0920 05:48:48.619649 5070 addon.go:266] installing PKI for "test" --- PASS: Test_InstallPKI (2.19s) === RUN Test_IsPKIInstalled --- PASS: Test_IsPKIInstalled (0.00s) PASS ok k8s.io/kops/channels/pkg/channels 2.304s === RUN TestGetUpdates --- PASS: TestGetUpdates (0.00s) PASS ok k8s.io/kops/channels/pkg/cmd 0.175s ? k8s.io/kops/cloudmock/aws/mockautoscaling [no test files] ? k8s.io/kops/cloudmock/aws/mockec2 [no test files] ? k8s.io/kops/cloudmock/aws/mockelb [no test files] ? k8s.io/kops/cloudmock/aws/mockelbv2 [no test files] ? k8s.io/kops/cloudmock/aws/mockeventbridge [no test files] ? k8s.io/kops/cloudmock/aws/mockiam [no test files] ? k8s.io/kops/cloudmock/aws/mockroute53 [no test files] ? k8s.io/kops/cloudmock/aws/mocksqs [no test files] ? k8s.io/kops/cloudmock/gce [no test files] ? k8s.io/kops/cloudmock/gce/gcphttp [no test files] ? k8s.io/kops/cloudmock/gce/mockcloudresourcemanager [no test files] ? k8s.io/kops/cloudmock/gce/mockcompute [no test files] ? k8s.io/kops/cloudmock/gce/mockdns [no test files] ? k8s.io/kops/cloudmock/gce/mockiam [no test files] ? k8s.io/kops/cloudmock/gce/mockstorage [no test files] ? k8s.io/kops/cloudmock/openstack [no test files] ? k8s.io/kops/cloudmock/openstack/mockblockstorage [no test files] ? k8s.io/kops/cloudmock/openstack/mockcompute [no test files] ? k8s.io/kops/cloudmock/openstack/mockdns [no test files] ? k8s.io/kops/cloudmock/openstack/mockimage [no test files] ? k8s.io/kops/cloudmock/openstack/mockloadbalancer [no test files] ? k8s.io/kops/cloudmock/openstack/mocknetworking [no test files] === RUN TestCreateClusterMinimal I0920 05:49:02.370781 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.371088 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:02.371150 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:02.371209 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:02.371301 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:02.371466 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:02.371495 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:02.371543 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:02.371578 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:02.371694 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:02.371743 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:02.425776 9338 new_cluster.go:1277] Cloud Provider ID = aws I0920 05:49:02.426028 9338 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0920 05:49:02.426514 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:02.442618 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.442800 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:02.442840 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:02.442876 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:02.442915 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:02.442946 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:02.442969 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:02.442998 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:02.443023 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:02.443105 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:02.443141 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:02.487747 9338 new_cluster.go:1277] Cloud Provider ID = aws I0920 05:49:02.487984 9338 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0920 05:49:02.488363 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:02.496000 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.496174 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:02.496218 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:02.496252 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:02.496293 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:02.496323 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:02.496345 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:02.496385 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:02.496408 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:02.496484 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:02.496516 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:02.540834 9338 new_cluster.go:1277] Cloud Provider ID = aws I0920 05:49:02.541072 9338 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0920 05:49:02.541448 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:02.551068 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.551268 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:02.551315 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:02.551365 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:02.551409 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:02.551446 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:02.551471 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:02.551503 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:02.551526 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:02.551618 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:02.551653 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:02.631034 9338 new_cluster.go:1277] Cloud Provider ID = aws I0920 05:49:02.631182 9338 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0920 05:49:02.631425 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:02.636342 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.636474 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:02.636506 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:02.636531 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:02.636558 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:02.636582 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:02.636597 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:02.636618 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:02.636663 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:02.636739 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:02.636776 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:02.666200 9338 new_cluster.go:1277] Cloud Provider ID = aws I0920 05:49:02.666348 9338 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0920 05:49:02.666643 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:02.671260 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.671395 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:02.671423 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:02.671445 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:02.671470 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:02.671492 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:02.671505 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:02.671526 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:02.671542 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:02.671594 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:02.671617 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:02.726623 9338 new_cluster.go:1277] Cloud Provider ID = aws I0920 05:49:02.726823 9338 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0920 05:49:02.727088 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:02.732187 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.732330 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:02.732372 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:02.732398 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:02.732427 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:02.732453 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:02.732467 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:02.732487 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:02.732503 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:02.732589 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:02.732612 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:02.767914 9338 new_cluster.go:1277] Cloud Provider ID = aws I0920 05:49:02.768058 9338 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0920 05:49:02.768299 9338 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterMinimal (0.40s) === RUN TestCreateClusterHetzner I0920 05:49:02.773232 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.773441 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:02.773477 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:02.773504 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:02.773531 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:02.773592 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:02.773609 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:02.773630 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:02.773645 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:02.773705 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:02.773728 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0920 05:49:02.854470 9338 new_cluster.go:886] Running with masters in the same AZs; redundancy will be reduced I0920 05:49:02.854522 9338 new_cluster.go:1277] Cloud Provider ID = hetzner I0920 05:49:02.860426 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.860588 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:02.860628 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:02.860656 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:02.860682 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:02.860706 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:02.860719 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:02.860740 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:02.860755 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:02.860809 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:02.860840 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:02.900676 9338 new_cluster.go:1277] Cloud Provider ID = hetzner --- PASS: TestCreateClusterHetzner (0.13s) === RUN TestCreateClusterOverride I0920 05:49:02.908071 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.908301 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:02.908347 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:02.908386 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:02.908425 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:02.908495 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:02.908519 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:02.908553 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:02.908579 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:02.908663 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:02.908696 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:02.981943 9338 new_cluster.go:1277] Cloud Provider ID = aws I0920 05:49:02.982657 9338 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0920 05:49:02.983021 9338 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterOverride (0.08s) === RUN TestCreateClusterComplex I0920 05:49:02.991582 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.991734 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:02.991773 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:02.991810 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:02.991845 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:02.991909 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:02.991930 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:02.991958 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:02.991978 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:02.992050 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:02.992080 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.079089 9338 new_cluster.go:1277] Cloud Provider ID = aws I0920 05:49:03.079237 9338 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0920 05:49:03.079482 9338 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterComplex (0.09s) === RUN TestCreateClusterHA I0920 05:49:03.084429 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.084628 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.084672 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.084711 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.084755 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.084830 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.084856 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.084893 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.084919 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.085013 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.085061 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.147987 9338 new_cluster.go:1277] Cloud Provider ID = aws I0920 05:49:03.148258 9338 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0920 05:49:03.148271 9338 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0920 05:49:03.148279 9338 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0920 05:49:03.148554 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:03.157535 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.157684 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.157715 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.157742 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.157769 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.157792 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.157806 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.157829 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.157844 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.157901 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.157925 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.185587 9338 new_cluster.go:1277] Cloud Provider ID = aws I0920 05:49:03.185860 9338 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0920 05:49:03.185873 9338 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0920 05:49:03.185880 9338 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0920 05:49:03.186161 9338 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHA (0.11s) === RUN TestCreateClusterHAGCE I0920 05:49:03.193096 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.193255 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.193286 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.193321 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.193373 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.193441 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.193456 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.193474 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.193488 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.193543 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.193565 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.244893 9338 new_cluster.go:1277] Cloud Provider ID = gce I0920 05:49:03.245025 9338 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterHAGCE (0.06s) === RUN TestCreateClusterGCE I0920 05:49:03.253403 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.253547 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.253577 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.253601 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.253628 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.253672 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.253686 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.253705 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.253720 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.253771 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.253793 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.284098 9338 new_cluster.go:528] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0920 05:49:03.284181 9338 new_cluster.go:1277] Cloud Provider ID = gce I0920 05:49:03.284245 9338 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterGCE (0.04s) === RUN TestCreateClusterHASharedZone I0920 05:49:03.288809 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.288941 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.288970 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.288992 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.289017 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.289062 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.289076 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.289095 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.289110 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.289165 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.289188 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0920 05:49:03.314766 9338 new_cluster.go:886] Running with masters in the same AZs; redundancy will be reduced I0920 05:49:03.314817 9338 new_cluster.go:1277] Cloud Provider ID = aws I0920 05:49:03.315000 9338 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0920 05:49:03.315247 9338 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZone (0.03s) === RUN TestCreateClusterHASharedZones I0920 05:49:03.321183 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.321317 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.321347 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.321369 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.321395 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.321441 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.321454 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.321472 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.321489 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.321543 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.321564 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0920 05:49:03.356805 9338 new_cluster.go:886] Running with masters in the same AZs; redundancy will be reduced I0920 05:49:03.356893 9338 new_cluster.go:1277] Cloud Provider ID = aws I0920 05:49:03.357272 9338 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0920 05:49:03.357291 9338 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0920 05:49:03.357611 9338 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZones (0.04s) === RUN TestCreateClusterPrivate I0920 05:49:03.366198 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.366370 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.366411 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.366447 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.366486 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.366544 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.366570 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.366606 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.366628 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.366715 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.366747 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.396841 9338 new_cluster.go:1277] Cloud Provider ID = aws I0920 05:49:03.397042 9338 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0920 05:49:03.397061 9338 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0920 05:49:03.397337 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:03.397372 9338 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterPrivate (0.04s) === RUN TestCreateClusterPrivateGCE I0920 05:49:03.402724 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.402880 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.402920 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.402952 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.402990 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.403043 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.403068 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.403103 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.403124 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.403190 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.403217 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.454537 9338 new_cluster.go:528] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0920 05:49:03.454617 9338 new_cluster.go:1277] Cloud Provider ID = gce I0920 05:49:03.454708 9338 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterPrivateGCE (0.06s) === RUN TestCreateClusterWithNGWSpecified I0920 05:49:03.462312 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.462456 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.462495 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.462520 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.462547 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.462589 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.462604 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.462625 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.462638 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.462688 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.462717 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.487927 9338 new_cluster.go:1277] Cloud Provider ID = aws I0920 05:49:03.488110 9338 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0920 05:49:03.488122 9338 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0920 05:49:03.488402 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:03.488428 9338 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithNGWSpecified (0.03s) === RUN TestCreateClusterWithINGWSpecified I0920 05:49:03.493601 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.493751 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.493790 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.493812 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.493837 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.493888 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.493901 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.493928 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.493941 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.494002 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.494023 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.529083 9338 new_cluster.go:1277] Cloud Provider ID = aws I0920 05:49:03.529270 9338 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0920 05:49:03.529283 9338 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0920 05:49:03.529558 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:03.529592 9338 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithINGWSpecified (0.04s) === RUN TestCreateClusterSharedVPC I0920 05:49:03.534997 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.535133 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.535170 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.535195 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.535221 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.535273 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.535287 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.535316 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.535329 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.535382 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.535411 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.624023 9338 new_cluster.go:1277] Cloud Provider ID = aws I0920 05:49:03.624196 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0920 05:49:03.624239 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0920 05:49:03.624281 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0920 05:49:03.624297 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0920 05:49:03.624347 9338 subnets.go:185] Assigned CIDR 10.2.0.0/15 to subnet us-test-1a I0920 05:49:03.624635 9338 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedVPC (0.09s) === RUN TestCreateClusterSharedSubnets I0920 05:49:03.629069 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.629198 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.629224 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.629244 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.629266 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.629307 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.629321 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.629340 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.629354 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.629404 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.629424 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.684691 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0920 05:49:03.684736 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0920 05:49:03.684794 9338 new_cluster.go:1277] Cloud Provider ID = aws I0920 05:49:03.684920 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0920 05:49:03.684937 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0920 05:49:03.684956 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0920 05:49:03.684968 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0920 05:49:03.685258 9338 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnets (0.06s) === RUN TestCreateClusterSharedSubnetsVpcLookup I0920 05:49:03.691156 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.691289 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.691327 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.691352 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.691379 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.691429 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.691443 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.691463 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.691485 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.691534 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.691562 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.729257 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:03.729309 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0920 05:49:03.729327 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0920 05:49:03.729385 9338 new_cluster.go:1277] Cloud Provider ID = aws I0920 05:49:03.729548 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0920 05:49:03.729566 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0920 05:49:03.729593 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0920 05:49:03.729605 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0920 05:49:03.729886 9338 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnetsVpcLookup (0.04s) === RUN TestCreateClusterPrivateSharedSubnets I0920 05:49:03.734398 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.734525 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.734563 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.734585 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.734609 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.734659 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.734672 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.734690 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.734703 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.734754 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.734774 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.794590 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0920 05:49:03.794638 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0920 05:49:03.794698 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0920 05:49:03.794713 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0920 05:49:03.794736 9338 new_cluster.go:1277] Cloud Provider ID = aws I0920 05:49:03.794861 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0920 05:49:03.794877 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0920 05:49:03.794896 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0920 05:49:03.794908 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0920 05:49:03.795193 9338 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterPrivateSharedSubnets (0.07s) === RUN TestCreateClusterIPv6 I0920 05:49:03.799911 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.800091 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.800131 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.800154 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.800179 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.800228 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.800244 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.800262 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.800284 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.800812 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.800851 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.819062 9338 new_cluster.go:1277] Cloud Provider ID = aws I0920 05:49:03.819199 9338 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0920 05:49:03.819519 9338 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterIPv6 (0.02s) === RUN TestCreateClusterDifferentAMIs I0920 05:49:03.824783 9338 featureflag.go:160] FeatureFlag "APIServerNodes"=true I0920 05:49:03.824797 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.824924 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.824961 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.824982 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.825007 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.825056 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.825069 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.825088 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.825111 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.825160 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.825189 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.861844 9338 new_cluster.go:1277] Cloud Provider ID = aws I0920 05:49:03.861918 9338 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0920 05:49:03.861929 9338 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0920 05:49:03.862183 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:03.868065 9338 featureflag.go:160] FeatureFlag "APIServerNodes"=false --- PASS: TestCreateClusterDifferentAMIs (0.04s) === RUN TestCreateClusterKarpenter I0920 05:49:03.868145 9338 featureflag.go:160] FeatureFlag "Karpenter"=true I0920 05:49:03.868154 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.868285 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.868315 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.868337 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.868362 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.868407 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.868423 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.868441 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.868455 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.868505 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.868526 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0920 05:49:03.907762 9338 new_cluster.go:1277] Cloud Provider ID = aws I0920 05:49:03.907937 9338 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0920 05:49:03.907954 9338 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0920 05:49:03.907970 9338 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0920 05:49:03.908281 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:03.913652 9338 featureflag.go:160] FeatureFlag "Karpenter"=false --- PASS: TestCreateClusterKarpenter (0.05s) === RUN TestParseCloudLabels --- PASS: TestParseCloudLabels (0.00s) === RUN TestConfirmation --- PASS: TestConfirmation (0.00s) === RUN TestEditInstanceGroup I0920 05:49:03.913974 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.914119 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.914155 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.914215 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.914253 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.914282 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.914308 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.914341 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.914363 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.918659 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2021-02-05"] }], Owners: ["383156758163"] } --- PASS: TestEditInstanceGroup (0.01s) === RUN TestMinimal I0920 05:49:03.919604 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.919732 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.919758 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.919789 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.919819 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.919860 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.919883 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.919909 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.919927 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.930840 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:03.932418 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:03.932478 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0920 05:49:04.841684 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:06.326487 9338 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0920 05:49:06.327237 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:06.328381 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:06.328776 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:06.329648 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:06.329693 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:06.329874 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0920 05:49:06.330032 9338 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0920 05:49:06.335414 9338 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0920 05:49:06.336367 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:06.336778 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:06.337765 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:06.340888 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:06.341077 9338 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0920 05:49:06.341648 9338 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0920 05:49:06.341990 9338 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0920 05:49:06.382195 9338 target.go:112] Terraform output is in /tmp/test2821003159/out I0920 05:49:06.384434 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:06.384571 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:06.384615 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:06.384648 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:06.384684 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:06.384719 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:06.384737 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:06.384762 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:06.384783 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:06.400351 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:06.402968 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:06.403056 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0920 05:49:06.439873 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0920 05:49:06.510481 9338 executor.go:111] Tasks: 0 done / 87 total; 43 can run W0920 05:49:06.510929 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0920 05:49:06.511074 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:06.511137 9338 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0920 05:49:06.511156 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:06.511231 9338 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0920 05:49:06.511222 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:06.511344 9338 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0920 05:49:06.511362 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:06.511438 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0920 05:49:06.512515 9338 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0920 05:49:06.515784 9338 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0920 05:49:06.516123 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:06.516258 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:06.516903 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:06.517509 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:06.517595 9338 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0920 05:49:06.517775 9338 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0920 05:49:06.517932 9338 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0920 05:49:06.522900 9338 target.go:168] Cloudformation output is in /tmp/test422793289/out --- PASS: TestMinimal (2.61s) === RUN TestMinimal_v1_23 I0920 05:49:06.528497 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:06.528647 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:06.528684 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:06.528749 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:06.528783 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:06.528811 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:06.528830 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:06.528852 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:06.528870 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:06.552345 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I0920 05:49:06.555660 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:06.555757 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:06.595717 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:06.693479 9338 executor.go:111] Tasks: 0 done / 92 total; 45 can run W0920 05:49:06.693787 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:06.694010 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:06.694041 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:06.694037 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:06.694127 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:06.694247 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:06.695933 9338 executor.go:111] Tasks: 45 done / 92 total; 19 can run I0920 05:49:06.698952 9338 executor.go:111] Tasks: 64 done / 92 total; 24 can run I0920 05:49:06.699108 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:06.699314 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:06.700535 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:06.703426 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:06.703524 9338 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0920 05:49:06.703745 9338 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0920 05:49:06.703901 9338 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0920 05:49:06.735679 9338 target.go:112] Terraform output is in /tmp/test2655345283/out I0920 05:49:06.738135 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:06.738267 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:06.738309 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:06.738343 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:06.738376 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:06.738414 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:06.738432 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:06.738456 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:06.738484 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:06.753808 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:06.755645 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:06.755711 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0920 05:49:06.793345 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0920 05:49:06.886386 9338 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0920 05:49:06.886866 9338 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0920 05:49:06.886897 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:06.886899 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:06.887007 9338 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0920 05:49:06.887041 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:06.887061 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:06.887207 9338 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0920 05:49:06.887227 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:06.887339 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0920 05:49:06.888718 9338 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0920 05:49:06.893974 9338 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0920 05:49:06.894433 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:06.894603 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:06.895488 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:06.896868 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:06.897013 9338 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0920 05:49:06.897273 9338 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0920 05:49:06.897485 9338 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0920 05:49:06.900326 9338 target.go:168] Cloudformation output is in /tmp/test4030320387/out --- PASS: TestMinimal_v1_23 (0.38s) === RUN TestMinimal_v1_24 I0920 05:49:06.910265 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:06.910484 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:06.910535 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:06.910618 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:06.910667 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:06.910707 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:06.910738 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:06.910797 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:06.910841 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:06.940311 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0920 05:49:06.943885 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:06.943988 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:06.983765 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:07.157520 9338 executor.go:111] Tasks: 0 done / 93 total; 46 can run I0920 05:49:07.158007 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0920 05:49:07.158044 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:07.158045 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:07.158081 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:07.158123 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:07.158263 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:07.160465 9338 executor.go:111] Tasks: 46 done / 93 total; 19 can run I0920 05:49:07.167681 9338 executor.go:111] Tasks: 65 done / 93 total; 24 can run I0920 05:49:07.167929 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:07.168105 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:07.169609 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:07.170800 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:07.170925 9338 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0920 05:49:07.171233 9338 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0920 05:49:07.171483 9338 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0920 05:49:07.225829 9338 target.go:112] Terraform output is in /tmp/test1911824935/out I0920 05:49:07.230437 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:07.230738 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:07.230825 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:07.230896 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:07.230979 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:07.231031 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:07.231061 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:07.231100 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:07.231125 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:07.254652 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:07.258003 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:07.258105 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0920 05:49:07.296438 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0920 05:49:07.392231 9338 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0920 05:49:07.392886 9338 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0920 05:49:07.392929 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:07.392993 9338 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0920 05:49:07.393033 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0920 05:49:07.393066 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:07.393161 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:07.393267 9338 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0920 05:49:07.393282 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:07.393371 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0920 05:49:07.394749 9338 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0920 05:49:07.399797 9338 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0920 05:49:07.400081 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:07.400233 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:07.400965 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:07.404453 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:07.404578 9338 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0920 05:49:07.404922 9338 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0920 05:49:07.405159 9338 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0920 05:49:07.408058 9338 target.go:168] Cloudformation output is in /tmp/test3270013032/out --- PASS: TestMinimal_v1_24 (0.51s) === RUN TestMinimal_v1_25 I0920 05:49:07.418198 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:07.418516 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:07.418593 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:07.418735 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:07.418818 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:07.418882 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:07.418935 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:07.418996 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:07.419041 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:07.444169 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0920 05:49:07.448003 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:07.448092 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:07.490146 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:07.642235 9338 executor.go:111] Tasks: 0 done / 93 total; 46 can run I0920 05:49:07.642657 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0920 05:49:07.642692 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:07.642713 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:07.642697 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:07.642829 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0920 05:49:07.643240 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:07.646407 9338 executor.go:111] Tasks: 46 done / 93 total; 19 can run I0920 05:49:07.652916 9338 executor.go:111] Tasks: 65 done / 93 total; 24 can run I0920 05:49:07.653059 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:07.653249 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:07.653796 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:07.656473 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:07.656601 9338 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0920 05:49:07.656888 9338 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0920 05:49:07.657070 9338 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0920 05:49:07.694199 9338 target.go:112] Terraform output is in /tmp/test2229144811/out I0920 05:49:07.698861 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:07.699066 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:07.699119 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:07.699167 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:07.699220 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:07.699263 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:07.699292 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:07.699331 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:07.699357 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:07.725325 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:07.728711 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:07.728812 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0920 05:49:07.766909 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0920 05:49:07.843712 9338 executor.go:111] Tasks: 0 done / 87 total; 43 can run W0920 05:49:07.844238 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0920 05:49:07.844357 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:07.844463 9338 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0920 05:49:07.844502 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:07.844627 9338 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0920 05:49:07.844711 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:07.844890 9338 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0920 05:49:07.844915 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:07.845036 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0920 05:49:07.846853 9338 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0920 05:49:07.855627 9338 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0920 05:49:07.856182 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:07.856395 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:07.857831 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:07.859055 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:07.859268 9338 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0920 05:49:07.859649 9338 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0920 05:49:07.859862 9338 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0920 05:49:07.862856 9338 target.go:168] Cloudformation output is in /tmp/test2704280661/out --- PASS: TestMinimal_v1_25 (0.45s) === RUN TestMinimal_v1_26 I0920 05:49:07.872961 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:07.873181 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:07.873234 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:07.873278 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:07.873342 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:07.873446 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:07.873484 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:07.873531 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:07.873573 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:07.897455 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* This version of kubernetes is not yet supported; upgrading kops is required (you can bypass this check by exporting KOPS_RUN_TOO_NEW_VERSION) ********************************************************************************* I0920 05:49:07.900594 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:07.900675 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:07.940011 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:08.086448 9338 executor.go:111] Tasks: 0 done / 92 total; 45 can run I0920 05:49:08.086824 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:08.087001 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:08.087052 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:08.087176 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:08.087216 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:08.087354 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:08.088629 9338 executor.go:111] Tasks: 45 done / 92 total; 19 can run I0920 05:49:08.095331 9338 executor.go:111] Tasks: 64 done / 92 total; 24 can run I0920 05:49:08.095667 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:08.095844 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:08.097149 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:08.097491 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:08.097566 9338 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0920 05:49:08.097848 9338 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0920 05:49:08.098008 9338 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0920 05:49:08.142945 9338 target.go:112] Terraform output is in /tmp/test1866913858/out I0920 05:49:08.145465 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:08.145591 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:08.145631 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:08.145663 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:08.145697 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:08.145725 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:08.145742 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:08.145767 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:08.145782 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:08.159142 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:08.160990 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:08.161051 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0920 05:49:08.197605 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0920 05:49:08.262951 9338 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0920 05:49:08.263647 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:08.263793 9338 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key W0920 05:49:08.263816 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0920 05:49:08.263690 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:08.263833 9338 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0920 05:49:08.263809 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:08.263869 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:08.263907 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0920 05:49:08.263997 9338 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0920 05:49:08.265509 9338 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0920 05:49:08.268430 9338 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0920 05:49:08.268754 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:08.269675 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:08.271416 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:08.273776 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:08.273862 9338 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0920 05:49:08.274221 9338 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0920 05:49:08.274463 9338 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0920 05:49:08.277557 9338 target.go:168] Cloudformation output is in /tmp/test4014535900/out --- PASS: TestMinimal_v1_26 (0.42s) === RUN TestHetzner I0920 05:49:08.288323 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0920 05:49:08.356062 9338 apply_cluster.go:487] Gossip DNS: skipping DNS validation I0920 05:49:08.432767 9338 executor.go:111] Tasks: 0 done / 50 total; 39 can run I0920 05:49:08.435497 9338 executor.go:111] Tasks: 39 done / 50 total; 5 can run I0920 05:49:08.436678 9338 executor.go:111] Tasks: 44 done / 50 total; 2 can run I0920 05:49:08.441180 9338 executor.go:111] Tasks: 46 done / 50 total; 4 can run I0920 05:49:08.443786 9338 executor.go:111] Tasks: 50 done / 50 total; 0 can run I0920 05:49:08.462367 9338 target.go:112] Terraform output is in /tmp/test2758210519/out --- PASS: TestHetzner (0.18s) === RUN TestNvidia I0920 05:49:08.465984 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:08.466410 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:08.466521 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:08.466779 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:08.466836 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:08.466882 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:08.466912 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:08.466949 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:08.466978 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:08.491563 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:08.494894 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:08.494990 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:08.533250 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:08.598670 9338 executor.go:111] Tasks: 0 done / 88 total; 44 can run W0920 05:49:08.599255 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:08.599388 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:08.599392 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:08.599458 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:08.599487 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:08.599596 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0920 05:49:08.604063 9338 executor.go:111] Tasks: 44 done / 88 total; 19 can run I0920 05:49:08.607033 9338 executor.go:111] Tasks: 63 done / 88 total; 21 can run I0920 05:49:08.607404 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:08.607566 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:08.608143 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:08.610648 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:08.610795 9338 executor.go:111] Tasks: 84 done / 88 total; 2 can run I0920 05:49:08.611083 9338 executor.go:111] Tasks: 86 done / 88 total; 2 can run I0920 05:49:08.611266 9338 executor.go:111] Tasks: 88 done / 88 total; 0 can run I0920 05:49:08.655637 9338 target.go:112] Terraform output is in /tmp/test3987202598/out I0920 05:49:08.657962 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:08.658091 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:08.658130 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:08.658161 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:08.658193 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:08.658221 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:08.658238 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:08.658261 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:08.658279 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:08.672416 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:08.674368 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:08.674427 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:08.711615 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0920 05:49:08.812411 9338 executor.go:111] Tasks: 0 done / 88 total; 44 can run W0920 05:49:08.812734 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0920 05:49:08.812915 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:08.813010 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:08.813156 9338 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0920 05:49:08.813043 9338 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0920 05:49:08.813175 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:08.813198 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:08.813256 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0920 05:49:08.813301 9338 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0920 05:49:08.818016 9338 executor.go:111] Tasks: 44 done / 88 total; 19 can run I0920 05:49:08.823497 9338 executor.go:111] Tasks: 63 done / 88 total; 21 can run I0920 05:49:08.823868 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:08.823991 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:08.824603 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:08.826368 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:08.826509 9338 executor.go:111] Tasks: 84 done / 88 total; 2 can run I0920 05:49:08.826744 9338 executor.go:111] Tasks: 86 done / 88 total; 2 can run I0920 05:49:08.826926 9338 executor.go:111] Tasks: 88 done / 88 total; 0 can run I0920 05:49:08.828845 9338 target.go:168] Cloudformation output is in /tmp/test2035212383/out --- PASS: TestNvidia (0.37s) === RUN TestMinimalGossip I0920 05:49:08.834692 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:08.834820 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:08.834851 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:08.834880 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:08.834910 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:08.834958 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:08.834975 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:08.834998 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:08.835015 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:08.862467 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:08.862565 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:08.900403 9338 apply_cluster.go:487] Gossip DNS: skipping DNS validation I0920 05:49:09.000777 9338 executor.go:111] Tasks: 0 done / 86 total; 42 can run W0920 05:49:09.001334 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:09.002504 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:09.002878 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:09.006857 9338 executor.go:111] Tasks: 42 done / 86 total; 19 can run I0920 05:49:09.011480 9338 executor.go:111] Tasks: 61 done / 86 total; 21 can run I0920 05:49:09.011853 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:09.011972 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:09.013155 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:09.013267 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:09.013403 9338 executor.go:111] Tasks: 82 done / 86 total; 2 can run I0920 05:49:09.013672 9338 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0920 05:49:09.013802 9338 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0920 05:49:09.053222 9338 target.go:112] Terraform output is in /tmp/test2910114068/out --- PASS: TestMinimalGossip (0.22s) === RUN TestMinimalGossipIRSA I0920 05:49:09.055525 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:09.055664 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:09.055704 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:09.055735 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:09.055767 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:09.055824 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:09.055842 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:09.055866 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:09.055883 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:09.072252 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:09.072314 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:09.110505 9338 apply_cluster.go:487] Gossip DNS: skipping DNS validation I0920 05:49:09.219431 9338 executor.go:111] Tasks: 0 done / 89 total; 44 can run I0920 05:49:09.221245 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:09.222063 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:09.223388 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:09.226420 9338 executor.go:111] Tasks: 44 done / 89 total; 20 can run I0920 05:49:09.231268 9338 executor.go:111] Tasks: 64 done / 89 total; 21 can run I0920 05:49:09.231471 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:09.231579 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:09.232192 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:09.232913 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:09.232991 9338 executor.go:111] Tasks: 85 done / 89 total; 2 can run I0920 05:49:09.233161 9338 executor.go:111] Tasks: 87 done / 89 total; 2 can run I0920 05:49:09.233257 9338 executor.go:111] Tasks: 89 done / 89 total; 0 can run I0920 05:49:09.269668 9338 target.go:112] Terraform output is in /tmp/test209713365/out --- PASS: TestMinimalGossipIRSA (0.22s) === RUN TestMinimalGCE I0920 05:49:09.272866 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:09.275392 9338 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W0920 05:49:09.481924 9338 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0920 05:49:09.482018 9338 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0920 05:49:09.482156 9338 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0920 05:49:09.482198 9338 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0920 05:49:09.482223 9338 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0920 05:49:09.482255 9338 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0920 05:49:09.482329 9338 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0920 05:49:09.484114 9338 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0920 05:49:09.490994 9338 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0920 05:49:09.494548 9338 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0920 05:49:09.496209 9338 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0920 05:49:09.496587 9338 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0920 05:49:09.520214 9338 target.go:112] Terraform output is in /tmp/test1196714189/out --- PASS: TestMinimalGCE (0.25s) === RUN TestMinimalGCEPrivate I0920 05:49:09.523911 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:09.525655 9338 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0920 05:49:09.663533 9338 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0920 05:49:09.663589 9338 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0920 05:49:09.663673 9338 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/main" W0920 05:49:09.663694 9338 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/events" W0920 05:49:09.663709 9338 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W0920 05:49:09.663728 9338 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W0920 05:49:09.663768 9338 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0920 05:49:09.664856 9338 executor.go:111] Tasks: 0 done / 66 total; 40 can run I0920 05:49:09.670469 9338 executor.go:111] Tasks: 40 done / 66 total; 18 can run I0920 05:49:09.673419 9338 executor.go:111] Tasks: 58 done / 66 total; 4 can run I0920 05:49:09.675670 9338 executor.go:111] Tasks: 62 done / 66 total; 3 can run I0920 05:49:09.677079 9338 executor.go:111] Tasks: 65 done / 66 total; 1 can run I0920 05:49:09.677223 9338 executor.go:111] Tasks: 66 done / 66 total; 0 can run I0920 05:49:09.708743 9338 target.go:112] Terraform output is in /tmp/test1358586403/out --- PASS: TestMinimalGCEPrivate (0.19s) === RUN TestMinimalGCEInternalLoadBalancer I0920 05:49:09.712639 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:09.714603 9338 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0920 05:49:09.858127 9338 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0920 05:49:09.858172 9338 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0920 05:49:09.858254 9338 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/main" W0920 05:49:09.858274 9338 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/events" W0920 05:49:09.858288 9338 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W0920 05:49:09.858307 9338 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W0920 05:49:09.858345 9338 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0920 05:49:09.858372 9338 autoscalinggroup.go:284] Not hooking the instance group manager up to anything. I0920 05:49:09.859535 9338 executor.go:111] Tasks: 0 done / 68 total; 41 can run I0920 05:49:09.865689 9338 executor.go:111] Tasks: 41 done / 68 total; 17 can run W0920 05:49:09.869080 9338 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0920 05:49:09.869106 9338 executor.go:111] Tasks: 58 done / 68 total; 4 can run I0920 05:49:09.872128 9338 executor.go:111] Tasks: 62 done / 68 total; 3 can run I0920 05:49:09.872962 9338 executor.go:111] Tasks: 65 done / 68 total; 2 can run I0920 05:49:09.873294 9338 executor.go:111] Tasks: 67 done / 68 total; 1 can run I0920 05:49:09.873618 9338 executor.go:111] Tasks: 68 done / 68 total; 0 can run I0920 05:49:09.901751 9338 target.go:112] Terraform output is in /tmp/test2371513319/out --- PASS: TestMinimalGCEInternalLoadBalancer (0.19s) === RUN TestMinimalGCELongClusterName I0920 05:49:09.905760 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:09.907904 9338 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W0920 05:49:10.111062 9338 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0920 05:49:10.111165 9338 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0920 05:49:10.111287 9338 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W0920 05:49:10.111321 9338 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W0920 05:49:10.111347 9338 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0920 05:49:10.111380 9338 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0920 05:49:10.111458 9338 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0920 05:49:10.112986 9338 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0920 05:49:10.116003 9338 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0920 05:49:10.119506 9338 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0920 05:49:10.121188 9338 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0920 05:49:10.121463 9338 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0920 05:49:10.151807 9338 target.go:112] Terraform output is in /tmp/test3869460073/out --- PASS: TestMinimalGCELongClusterName (0.25s) === RUN TestMinimalGCEInternalLoadBalancerLongClusterName I0920 05:49:10.155626 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:10.157589 9338 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W0920 05:49:10.352116 9338 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0920 05:49:10.352187 9338 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0920 05:49:10.352290 9338 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0920 05:49:10.352342 9338 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W0920 05:49:10.352363 9338 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W0920 05:49:10.352382 9338 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0920 05:49:10.352437 9338 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0920 05:49:10.352477 9338 autoscalinggroup.go:284] Not hooking the instance group manager up to anything. I0920 05:49:10.353906 9338 executor.go:111] Tasks: 0 done / 67 total; 42 can run I0920 05:49:10.360875 9338 executor.go:111] Tasks: 42 done / 67 total; 16 can run W0920 05:49:10.364405 9338 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0920 05:49:10.364444 9338 executor.go:111] Tasks: 58 done / 67 total; 5 can run I0920 05:49:10.367345 9338 executor.go:111] Tasks: 63 done / 67 total; 2 can run I0920 05:49:10.367515 9338 executor.go:111] Tasks: 65 done / 67 total; 1 can run I0920 05:49:10.367623 9338 executor.go:111] Tasks: 66 done / 67 total; 1 can run I0920 05:49:10.367708 9338 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0920 05:49:10.394145 9338 target.go:112] Terraform output is in /tmp/test1071863292/out --- PASS: TestMinimalGCEInternalLoadBalancerLongClusterName (0.24s) === RUN TestHA I0920 05:49:10.398478 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:10.398767 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:10.398837 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:10.398918 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:10.399008 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:10.399143 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:10.399203 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:10.399272 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:10.399320 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:10.431291 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:10.434660 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:10.434752 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:10.434825 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:10.434896 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:10.473251 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:10.574736 9338 executor.go:111] Tasks: 0 done / 111 total; 53 can run W0920 05:49:10.575181 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:10.575253 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.ha.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:10.575333 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:10.577345 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:10.577462 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:10.577729 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:10.577995 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:10.578587 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:10.581345 9338 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0920 05:49:10.586593 9338 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0920 05:49:10.586845 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:10.586926 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:10.587127 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:10.587340 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:10.589289 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:10.591992 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:10.592254 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:10.592844 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:10.592995 9338 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0920 05:49:10.593487 9338 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0920 05:49:10.593773 9338 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0920 05:49:10.667717 9338 target.go:112] Terraform output is in /tmp/test2948799772/out --- PASS: TestHA (0.27s) === RUN TestHighAvailabilityGCE I0920 05:49:10.672891 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:10.675036 9338 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0920 05:49:10.847944 9338 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0920 05:49:10.848014 9338 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0920 05:49:10.848092 9338 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/main" W0920 05:49:10.848112 9338 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/events" W0920 05:49:10.848126 9338 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W0920 05:49:10.848150 9338 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W0920 05:49:10.848190 9338 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0920 05:49:10.848238 9338 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0920 05:49:10.848277 9338 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0920 05:49:10.849627 9338 executor.go:111] Tasks: 0 done / 83 total; 48 can run I0920 05:49:10.852220 9338 executor.go:111] Tasks: 48 done / 83 total; 20 can run I0920 05:49:10.857346 9338 executor.go:111] Tasks: 68 done / 83 total; 7 can run I0920 05:49:10.862486 9338 executor.go:111] Tasks: 75 done / 83 total; 5 can run I0920 05:49:10.863367 9338 executor.go:111] Tasks: 80 done / 83 total; 3 can run I0920 05:49:10.863606 9338 executor.go:111] Tasks: 83 done / 83 total; 0 can run I0920 05:49:10.903304 9338 target.go:112] Terraform output is in /tmp/test3537230143/out --- PASS: TestHighAvailabilityGCE (0.23s) === RUN TestComplex I0920 05:49:10.907969 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:10.908203 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:10.908264 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:10.908375 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:10.908432 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:10.908479 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:10.908508 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:10.908545 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:10.908576 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:10.940440 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0920 05:49:10.944645 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:10.944737 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:10.984293 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:11.165959 9338 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0920 05:49:11.166412 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:11.166450 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:11.166545 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:11.166777 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:11.166923 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:11.168684 9338 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0920 05:49:11.173837 9338 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0920 05:49:11.174271 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:11.176343 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:11.177183 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:11.179186 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:11.179323 9338 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0920 05:49:11.179667 9338 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0920 05:49:11.179822 9338 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0920 05:49:11.238128 9338 target.go:112] Terraform output is in /tmp/test2181755173/out I0920 05:49:11.242462 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:11.242664 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:11.242720 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:11.242772 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:11.242825 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:11.242865 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:11.242895 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:11.242931 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:11.242959 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:11.277346 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0920 05:49:11.282594 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:11.282703 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:11.320975 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0920 05:49:11.475927 9338 executor.go:111] Tasks: 0 done / 122 total; 51 can run W0920 05:49:11.476230 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0920 05:49:11.476674 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:11.476836 9338 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0920 05:49:11.476866 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:11.476949 9338 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0920 05:49:11.480894 9338 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0920 05:49:11.487169 9338 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0920 05:49:11.487482 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:11.487625 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:11.488598 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:11.491262 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:11.491448 9338 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0920 05:49:11.491742 9338 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0920 05:49:11.491860 9338 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0920 05:49:11.496805 9338 target.go:168] Cloudformation output is in /tmp/test4230940578/out I0920 05:49:11.510785 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:11.511023 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:11.511079 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:11.511129 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:11.511183 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:11.511224 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:11.511261 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:11.511298 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:11.511328 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:11.547635 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0920 05:49:11.552263 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:11.552376 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:11.589809 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:11.752707 9338 executor.go:111] Tasks: 0 done / 122 total; 51 can run W0920 05:49:11.753324 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:11.753445 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0920 05:49:11.753467 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:11.753467 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:11.753536 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0920 05:49:11.756051 9338 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0920 05:49:11.764786 9338 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0920 05:49:11.765082 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:11.765384 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:11.766207 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:11.768063 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:11.768198 9338 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0920 05:49:11.768653 9338 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0920 05:49:11.768874 9338 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0920 05:49:11.836938 9338 target.go:112] Terraform output is in /tmp/test847648664/out --- PASS: TestComplex (0.93s) === RUN TestCompress I0920 05:49:11.840497 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:11.840721 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:11.840771 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:11.840820 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:11.840871 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:11.840953 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:11.840985 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:11.841021 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:11.841049 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:11.865558 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:11.867467 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:11.867537 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:11.905831 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:11.975561 9338 executor.go:111] Tasks: 0 done / 86 total; 42 can run W0920 05:49:11.975818 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:11.975964 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:11.976292 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:11.976333 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:11.976416 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0920 05:49:11.978385 9338 executor.go:111] Tasks: 42 done / 86 total; 19 can run I0920 05:49:11.982255 9338 executor.go:111] Tasks: 61 done / 86 total; 21 can run I0920 05:49:11.982672 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:11.982905 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:11.987330 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:11.990476 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:11.990585 9338 executor.go:111] Tasks: 82 done / 86 total; 2 can run I0920 05:49:11.990952 9338 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0920 05:49:11.991122 9338 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0920 05:49:12.023675 9338 target.go:112] Terraform output is in /tmp/test712017995/out --- PASS: TestCompress (0.19s) === RUN TestExternalPolicies I0920 05:49:12.026487 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:12.026688 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:12.026740 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:12.026785 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:12.026828 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:12.026905 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:12.026928 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:12.026958 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:12.026979 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:12.050416 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:12.053149 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:12.053232 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:12.090545 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:12.180726 9338 executor.go:111] Tasks: 0 done / 100 total; 45 can run W0920 05:49:12.181041 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:12.181078 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:12.181095 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externalpolicies.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:12.181094 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:12.181183 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:12.181367 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:12.183965 9338 executor.go:111] Tasks: 45 done / 100 total; 20 can run I0920 05:49:12.187344 9338 executor.go:111] Tasks: 65 done / 100 total; 30 can run I0920 05:49:12.187738 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:12.187961 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:12.189062 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:12.189140 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:12.189219 9338 executor.go:111] Tasks: 95 done / 100 total; 3 can run I0920 05:49:12.190445 9338 executor.go:111] Tasks: 98 done / 100 total; 2 can run I0920 05:49:12.190591 9338 executor.go:111] Tasks: 100 done / 100 total; 0 can run I0920 05:49:12.245238 9338 target.go:112] Terraform output is in /tmp/test1131393617/out --- PASS: TestExternalPolicies (0.22s) === RUN TestMinimalIPv6 I0920 05:49:12.247734 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:12.247878 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:12.247920 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:12.247986 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:12.248025 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:12.248056 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:12.248074 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:12.248105 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:12.248124 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:12.262716 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:12.264911 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:12.264973 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:12.302639 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:12.459052 9338 executor.go:111] Tasks: 0 done / 106 total; 45 can run W0920 05:49:12.459424 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:12.459464 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:12.459590 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:12.459842 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:12.459876 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:12.459967 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0920 05:49:12.462391 9338 executor.go:111] Tasks: 45 done / 106 total; 21 can run I0920 05:49:12.465519 9338 executor.go:111] Tasks: 66 done / 106 total; 30 can run I0920 05:49:12.465944 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:12.466076 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:12.466675 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:12.468651 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:12.468806 9338 executor.go:111] Tasks: 96 done / 106 total; 5 can run I0920 05:49:12.469236 9338 executor.go:111] Tasks: 101 done / 106 total; 5 can run I0920 05:49:12.469599 9338 executor.go:111] Tasks: 106 done / 106 total; 0 can run I0920 05:49:12.530759 9338 target.go:112] Terraform output is in /tmp/test4190177065/out I0920 05:49:12.536430 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:12.536623 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:12.536673 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:12.536709 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:12.536744 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:12.536782 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:12.536801 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:12.536825 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:12.536843 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:12.556920 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:12.560796 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:12.560902 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:12.598132 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0920 05:49:12.763239 9338 executor.go:111] Tasks: 0 done / 106 total; 45 can run W0920 05:49:12.763618 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:12.764132 9338 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0920 05:49:12.764167 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:12.764161 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:12.764231 9338 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0920 05:49:12.764238 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:12.764369 9338 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0920 05:49:12.764386 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:12.764463 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0920 05:49:12.765843 9338 executor.go:111] Tasks: 45 done / 106 total; 21 can run I0920 05:49:12.768945 9338 executor.go:111] Tasks: 66 done / 106 total; 30 can run I0920 05:49:12.769467 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:12.769675 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:12.770172 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:12.772280 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:12.772453 9338 executor.go:111] Tasks: 96 done / 106 total; 5 can run I0920 05:49:12.772845 9338 executor.go:111] Tasks: 101 done / 106 total; 5 can run I0920 05:49:12.773201 9338 executor.go:111] Tasks: 106 done / 106 total; 0 can run I0920 05:49:12.778095 9338 target.go:168] Cloudformation output is in /tmp/test2317964838/out --- PASS: TestMinimalIPv6 (0.54s) === RUN TestMinimalIPv6Private I0920 05:49:12.790294 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:12.790550 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:12.790638 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:12.790718 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:12.790803 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:12.790934 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:12.790994 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:12.791061 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:12.791114 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:12.820608 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I0920 05:49:12.824326 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:12.824409 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:12.862099 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:12.991342 9338 executor.go:111] Tasks: 0 done / 124 total; 46 can run I0920 05:49:12.991962 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:12.991990 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:12.992070 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0920 05:49:12.992141 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:12.992150 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:12.992120 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:12.993900 9338 executor.go:111] Tasks: 46 done / 124 total; 23 can run I0920 05:49:13.001681 9338 executor.go:111] Tasks: 69 done / 124 total; 36 can run I0920 05:49:13.001961 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:13.002154 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:13.003467 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:13.003534 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:13.003588 9338 executor.go:111] Tasks: 105 done / 124 total; 11 can run I0920 05:49:13.004152 9338 executor.go:111] Tasks: 116 done / 124 total; 8 can run I0920 05:49:13.004883 9338 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0920 05:49:13.068498 9338 target.go:112] Terraform output is in /tmp/test1682474691/out --- PASS: TestMinimalIPv6Private (0.28s) === RUN TestMinimalIPv6Calico I0920 05:49:13.073353 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:13.073579 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:13.073637 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:13.073740 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:13.073792 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:13.073834 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:13.073866 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:13.073904 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:13.073933 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:13.100785 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0920 05:49:13.102968 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:13.103032 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:13.140541 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:13.521697 9338 executor.go:111] Tasks: 0 done / 109 total; 47 can run W0920 05:49:13.521990 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:13.522349 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:13.522402 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:13.522654 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:13.522681 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:13.522766 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0920 05:49:13.527927 9338 executor.go:111] Tasks: 47 done / 109 total; 21 can run I0920 05:49:13.533398 9338 executor.go:111] Tasks: 68 done / 109 total; 31 can run I0920 05:49:13.533807 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:13.533995 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:13.534541 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:13.535853 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:13.535935 9338 executor.go:111] Tasks: 99 done / 109 total; 5 can run I0920 05:49:13.536345 9338 executor.go:111] Tasks: 104 done / 109 total; 5 can run I0920 05:49:13.536668 9338 executor.go:111] Tasks: 109 done / 109 total; 0 can run I0920 05:49:13.602410 9338 target.go:112] Terraform output is in /tmp/test4257890978/out --- PASS: TestMinimalIPv6Calico (0.53s) === RUN TestMinimalIPv6Cilium I0920 05:49:13.605724 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:13.605875 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:13.605915 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:13.605981 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:13.606016 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:13.606048 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:13.606066 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:13.606090 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:13.606108 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:13.620663 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:13.622929 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:13.622989 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:13.661356 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:13.850729 9338 executor.go:111] Tasks: 0 done / 107 total; 46 can run W0920 05:49:13.851624 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:13.851858 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0920 05:49:13.851867 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:13.851909 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:13.852050 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0920 05:49:13.852128 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:13.855094 9338 executor.go:111] Tasks: 46 done / 107 total; 21 can run I0920 05:49:13.863934 9338 executor.go:111] Tasks: 67 done / 107 total; 30 can run I0920 05:49:13.864651 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:13.864877 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:13.866340 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:13.867540 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:13.867684 9338 executor.go:111] Tasks: 97 done / 107 total; 5 can run I0920 05:49:13.868132 9338 executor.go:111] Tasks: 102 done / 107 total; 5 can run I0920 05:49:13.868548 9338 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0920 05:49:13.941827 9338 target.go:112] Terraform output is in /tmp/test2159355671/out I0920 05:49:13.946790 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:13.946999 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:13.947056 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:13.947107 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:13.947164 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:13.947206 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:13.947236 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:13.947275 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:13.947306 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:13.972583 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:13.976602 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:13.976734 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:14.016762 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0920 05:49:14.244265 9338 executor.go:111] Tasks: 0 done / 107 total; 46 can run W0920 05:49:14.244933 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0920 05:49:14.245102 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:14.245124 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:14.245334 9338 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0920 05:49:14.245464 9338 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0920 05:49:14.246892 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:14.247009 9338 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0920 05:49:14.245360 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:14.247547 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0920 05:49:14.252106 9338 executor.go:111] Tasks: 46 done / 107 total; 21 can run I0920 05:49:14.258087 9338 executor.go:111] Tasks: 67 done / 107 total; 30 can run I0920 05:49:14.258412 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:14.258657 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:14.259617 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:14.261432 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:14.261599 9338 executor.go:111] Tasks: 97 done / 107 total; 5 can run I0920 05:49:14.262021 9338 executor.go:111] Tasks: 102 done / 107 total; 5 can run I0920 05:49:14.262382 9338 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0920 05:49:14.266101 9338 target.go:168] Cloudformation output is in /tmp/test862083451/out --- PASS: TestMinimalIPv6Cilium (0.67s) === RUN TestMinimalWarmPool I0920 05:49:14.277567 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:14.277808 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:14.277863 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:14.277914 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:14.277964 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:14.278049 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:14.278082 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:14.278122 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:14.278153 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:14.307466 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:14.311345 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:14.311449 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:14.350301 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:14.551046 9338 executor.go:111] Tasks: 0 done / 89 total; 45 can run I0920 05:49:14.551710 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0920 05:49:14.551823 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:14.551760 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:14.551899 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:14.552076 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:14.552192 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-warmpool.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:14.556083 9338 executor.go:111] Tasks: 45 done / 89 total; 19 can run I0920 05:49:14.565676 9338 executor.go:111] Tasks: 64 done / 89 total; 21 can run I0920 05:49:14.566245 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:14.566463 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:14.567817 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:14.569376 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:14.569504 9338 executor.go:111] Tasks: 85 done / 89 total; 2 can run I0920 05:49:14.570491 9338 executor.go:111] Tasks: 87 done / 89 total; 2 can run I0920 05:49:14.570698 9338 executor.go:111] Tasks: 89 done / 89 total; 0 can run I0920 05:49:14.625542 9338 target.go:112] Terraform output is in /tmp/test3924759958/out --- PASS: TestMinimalWarmPool (0.35s) === RUN TestMinimalEtcd I0920 05:49:14.629903 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:14.630175 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:14.630232 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:14.630340 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:14.630400 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:14.630442 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:14.630471 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:14.630506 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:14.630534 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:14.656774 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:14.659943 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:14.660072 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:14.699273 9338 zones.go:93] ListHostedZonesPages { } W0920 05:49:14.788910 9338 model.go:248] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W0920 05:49:14.788971 9338 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 10 W0920 05:49:14.790770 9338 model.go:248] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0920 05:49:14.801753 9338 executor.go:111] Tasks: 0 done / 87 total; 43 can run W0920 05:49:14.802220 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:14.802530 9338 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0920 05:49:14.802550 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:14.802568 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:14.802719 9338 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0920 05:49:14.802708 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:14.804321 9338 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0920 05:49:14.804354 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:14.804570 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-etcd.example.com" },{ Key: "KubernetesCluster", Value: "minimal-etcd.example.com" },{ Key: "kubernetes.io/cluster/minimal-etcd.example.com", Value: "owned" }] }] } I0920 05:49:14.809142 9338 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0920 05:49:14.814498 9338 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0920 05:49:14.815017 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:14.815272 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:14.816262 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:14.818358 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:14.818500 9338 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0920 05:49:14.818767 9338 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0920 05:49:14.819012 9338 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0920 05:49:14.822744 9338 target.go:168] Cloudformation output is in /tmp/test2686715884/out --- PASS: TestMinimalEtcd (0.20s) === RUN TestMinimalGp3 I0920 05:49:14.832704 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:14.832928 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:14.832979 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:14.833025 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:14.833077 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:14.833157 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:14.833187 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:14.833225 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:14.833253 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:14.862483 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:14.865879 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:14.865982 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:14.905755 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:15.009170 9338 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0920 05:49:15.009446 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:15.009675 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:15.009873 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:15.009927 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:15.009966 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:15.010229 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0920 05:49:15.012728 9338 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0920 05:49:15.017934 9338 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0920 05:49:15.018588 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.018819 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.019879 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.021019 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.021135 9338 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0920 05:49:15.022275 9338 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0920 05:49:15.022624 9338 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0920 05:49:15.071998 9338 target.go:112] Terraform output is in /tmp/test4074478678/out I0920 05:49:15.075816 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:15.076030 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:15.076091 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:15.076147 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:15.076200 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:15.076243 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:15.076274 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:15.076310 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:15.076341 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:15.104587 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:15.107941 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.108044 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.147130 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0920 05:49:15.245758 9338 executor.go:111] Tasks: 0 done / 87 total; 43 can run W0920 05:49:15.246447 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:15.246585 9338 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0920 05:49:15.246622 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:15.246738 9338 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0920 05:49:15.246856 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:15.246867 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:15.247065 9338 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0920 05:49:15.247097 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:15.247254 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0920 05:49:15.249065 9338 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0920 05:49:15.254526 9338 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0920 05:49:15.254968 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.255137 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.256258 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.257350 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.257486 9338 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0920 05:49:15.257744 9338 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0920 05:49:15.257936 9338 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0920 05:49:15.260918 9338 target.go:168] Cloudformation output is in /tmp/test178753888/out --- PASS: TestMinimalGp3 (0.44s) === RUN TestMinimalLongClusterName I0920 05:49:15.270845 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:15.271070 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:15.271125 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:15.271168 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:15.271218 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:15.271297 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:15.271327 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:15.271369 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:15.271401 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:15.300095 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:15.303443 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.303549 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0920 05:49:15.342910 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:15.441743 9338 executor.go:111] Tasks: 0 done / 87 total; 43 can run W0920 05:49:15.443253 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:15.443830 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:15.443788 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:15.444124 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:15.444173 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:15.444274 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0920 05:49:15.449267 9338 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0920 05:49:15.457577 9338 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0920 05:49:15.457794 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.457938 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.459403 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.460586 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.460713 9338 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0920 05:49:15.461079 9338 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0920 05:49:15.461260 9338 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0920 05:49:15.518652 9338 target.go:112] Terraform output is in /tmp/test2835448001/out I0920 05:49:15.522732 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:15.522960 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:15.523024 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:15.523081 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:15.523137 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:15.523196 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:15.523225 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:15.523265 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:15.523295 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:15.547061 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:15.550130 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.550215 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0920 05:49:15.589208 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0920 05:49:15.688909 9338 executor.go:111] Tasks: 0 done / 87 total; 43 can run W0920 05:49:15.689331 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0920 05:49:15.689648 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:15.689662 9338 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0920 05:49:15.689697 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:15.689804 9338 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0920 05:49:15.689907 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:15.690131 9338 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0920 05:49:15.690165 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:15.690307 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "this.is.truly.a.really.really.long.cluster-name.minimal.example.com" },{ Key: "KubernetesCluster", Value: "this.is.truly.a.really.really.long.cluster-name.minimal.example.com" },{ Key: "kubernetes.io/cluster/this.is.truly.a.really.really.long.cluster-name.minimal.example.com", Value: "owned" }] }] } I0920 05:49:15.694535 9338 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0920 05:49:15.702103 9338 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0920 05:49:15.702422 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.702675 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.703536 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.705332 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.705582 9338 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0920 05:49:15.705904 9338 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0920 05:49:15.706101 9338 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0920 05:49:15.709680 9338 target.go:168] Cloudformation output is in /tmp/test3393375265/out --- PASS: TestMinimalLongClusterName (0.45s) === RUN TestExistingIAMCloudformation I0920 05:49:15.719799 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:15.720036 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:15.720090 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:15.720182 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:15.720234 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:15.720279 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:15.720311 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:15.720352 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:15.720380 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:15.748510 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:15.751804 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.751912 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.790378 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0920 05:49:15.885902 9338 executor.go:111] Tasks: 0 done / 77 total; 41 can run W0920 05:49:15.886509 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:15.886594 9338 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0920 05:49:15.886625 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:15.886668 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0920 05:49:15.886792 9338 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key W0920 05:49:15.886809 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:15.886809 9338 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0920 05:49:15.886816 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:15.886918 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } I0920 05:49:15.889207 9338 executor.go:111] Tasks: 41 done / 77 total; 11 can run I0920 05:49:15.899611 9338 executor.go:111] Tasks: 52 done / 77 total; 21 can run I0920 05:49:15.899841 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.900267 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.901390 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.902263 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.902421 9338 executor.go:111] Tasks: 73 done / 77 total; 2 can run I0920 05:49:15.902806 9338 executor.go:111] Tasks: 75 done / 77 total; 2 can run I0920 05:49:15.903072 9338 executor.go:111] Tasks: 77 done / 77 total; 0 can run I0920 05:49:15.905379 9338 target.go:168] Cloudformation output is in /tmp/test48095685/out --- PASS: TestExistingIAMCloudformation (0.19s) === RUN TestExistingSG I0920 05:49:15.914262 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:15.914488 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:15.914537 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:15.914580 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:15.914629 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:15.914708 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:15.914737 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:15.914774 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:15.914803 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:15.951861 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:15.956291 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.956399 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.956482 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.956562 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:15.995234 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:16.112832 9338 executor.go:111] Tasks: 0 done / 146 total; 53 can run I0920 05:49:16.113116 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existingsg.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:16.113219 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:16.113671 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:16.113730 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0920 05:49:16.113864 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:16.113872 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:16.113692 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:16.113915 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:16.119892 9338 executor.go:111] Tasks: 53 done / 146 total; 26 can run I0920 05:49:16.125387 9338 executor.go:111] Tasks: 79 done / 146 total; 58 can run I0920 05:49:16.126057 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.126206 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.126404 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.126687 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.127232 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.129752 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.130854 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.131539 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.131914 9338 executor.go:111] Tasks: 137 done / 146 total; 5 can run I0920 05:49:16.134159 9338 executor.go:111] Tasks: 142 done / 146 total; 4 can run I0920 05:49:16.134425 9338 executor.go:111] Tasks: 146 done / 146 total; 0 can run I0920 05:49:16.228360 9338 target.go:112] Terraform output is in /tmp/test644589878/out --- PASS: TestExistingSG (0.32s) === RUN TestBastionAdditionalUserData I0920 05:49:16.233856 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:16.234079 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:16.234138 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:16.234190 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:16.234243 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:16.234332 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:16.234363 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:16.234402 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:16.234430 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:16.267291 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:16.271727 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.271833 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.271901 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.310586 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:16.417020 9338 executor.go:111] Tasks: 0 done / 122 total; 48 can run I0920 05:49:16.417428 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.bastionuserdata.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:16.417486 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:16.417844 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:16.417882 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:16.417919 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:16.417953 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:16.418086 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0920 05:49:16.419821 9338 executor.go:111] Tasks: 48 done / 122 total; 29 can run I0920 05:49:16.430387 9338 executor.go:111] Tasks: 77 done / 122 total; 36 can run I0920 05:49:16.431087 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.431293 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.431592 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.432377 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.432757 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.433955 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.434152 9338 executor.go:111] Tasks: 113 done / 122 total; 5 can run I0920 05:49:16.434558 9338 executor.go:111] Tasks: 118 done / 122 total; 4 can run I0920 05:49:16.434875 9338 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0920 05:49:16.510223 9338 target.go:112] Terraform output is in /tmp/test2027094728/out --- PASS: TestBastionAdditionalUserData (0.28s) === RUN TestPrivateWeave I0920 05:49:16.514925 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:16.515169 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:16.515231 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:16.515285 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:16.515341 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:16.515441 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:16.515471 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:16.515514 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:16.515545 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:16.548203 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:16.551866 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.551965 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.552029 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.591067 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:16.717384 9338 executor.go:111] Tasks: 0 done / 120 total; 47 can run I0920 05:49:16.718171 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:16.718210 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:16.718323 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:16.718482 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:16.718524 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:16.718600 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateweave.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:16.718657 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:16.720915 9338 executor.go:111] Tasks: 47 done / 120 total; 28 can run I0920 05:49:16.730720 9338 executor.go:111] Tasks: 75 done / 120 total; 36 can run I0920 05:49:16.731062 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.731249 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.731416 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.731533 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.732290 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.734098 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.734321 9338 executor.go:111] Tasks: 111 done / 120 total; 5 can run I0920 05:49:16.734735 9338 executor.go:111] Tasks: 116 done / 120 total; 4 can run I0920 05:49:16.735001 9338 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0920 05:49:16.810538 9338 target.go:112] Terraform output is in /tmp/test2985676432/out --- PASS: TestPrivateWeave (0.30s) === RUN TestPrivateFlannel I0920 05:49:16.814937 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:16.815176 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:16.815239 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:16.815346 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:16.815403 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:16.815450 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:16.815479 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:16.815516 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:16.815545 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:16.847986 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0920 05:49:16.851781 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.851885 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.851950 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:16.892660 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:17.091472 9338 executor.go:111] Tasks: 0 done / 123 total; 50 can run W0920 05:49:17.091746 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:17.091912 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:17.091888 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateflannel.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:17.091949 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:17.092097 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:17.092200 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:17.092237 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:17.100113 9338 executor.go:111] Tasks: 50 done / 123 total; 28 can run I0920 05:49:17.105702 9338 executor.go:111] Tasks: 78 done / 123 total; 36 can run I0920 05:49:17.106131 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:17.106392 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:17.106692 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:17.106839 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:17.108372 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:17.110096 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:17.110294 9338 executor.go:111] Tasks: 114 done / 123 total; 5 can run I0920 05:49:17.111018 9338 executor.go:111] Tasks: 119 done / 123 total; 4 can run I0920 05:49:17.111358 9338 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0920 05:49:17.189310 9338 target.go:112] Terraform output is in /tmp/test1522660130/out --- PASS: TestPrivateFlannel (0.38s) === RUN TestPrivateCalico I0920 05:49:17.194166 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:17.194400 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:17.194459 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:17.194568 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:17.194621 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:17.194668 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:17.194700 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:17.194738 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:17.194766 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:17.228414 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0920 05:49:17.232995 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:17.233099 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:17.233163 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:17.271280 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:17.734578 9338 executor.go:111] Tasks: 0 done / 127 total; 50 can run W0920 05:49:17.734987 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:17.735134 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:17.735167 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:17.735284 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:17.735307 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:17.735311 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:17.735447 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:17.738125 9338 executor.go:111] Tasks: 50 done / 127 total; 28 can run I0920 05:49:17.747533 9338 executor.go:111] Tasks: 78 done / 127 total; 40 can run I0920 05:49:17.748310 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:17.748520 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:17.748825 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:17.749041 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:17.749629 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:17.751503 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:17.751623 9338 executor.go:111] Tasks: 118 done / 127 total; 5 can run I0920 05:49:17.752191 9338 executor.go:111] Tasks: 123 done / 127 total; 4 can run I0920 05:49:17.752561 9338 executor.go:111] Tasks: 127 done / 127 total; 0 can run I0920 05:49:17.828669 9338 target.go:112] Terraform output is in /tmp/test3994266999/out --- PASS: TestPrivateCalico (0.64s) === RUN TestPrivateCilium I0920 05:49:17.834121 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:17.834362 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:17.834425 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:17.834534 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:17.834590 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:17.834636 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:17.834665 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:17.834703 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:17.834732 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:17.866760 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:17.871045 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:17.871150 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:17.871219 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:17.909160 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:18.056445 9338 executor.go:111] Tasks: 0 done / 120 total; 47 can run W0920 05:49:18.057157 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:18.057340 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:18.057375 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:18.057431 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:18.057414 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:18.057532 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:18.057554 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:18.063232 9338 executor.go:111] Tasks: 47 done / 120 total; 28 can run I0920 05:49:18.068902 9338 executor.go:111] Tasks: 75 done / 120 total; 36 can run I0920 05:49:18.069337 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:18.069653 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:18.069937 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:18.070533 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:18.070915 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:18.072826 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:18.072952 9338 executor.go:111] Tasks: 111 done / 120 total; 5 can run I0920 05:49:18.073380 9338 executor.go:111] Tasks: 116 done / 120 total; 4 can run I0920 05:49:18.073714 9338 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0920 05:49:18.148720 9338 target.go:112] Terraform output is in /tmp/test352587817/out I0920 05:49:18.153005 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:18.153214 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:18.153277 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:18.153358 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:18.153427 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:18.153479 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:18.153519 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:18.153578 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:18.153606 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:18.181961 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:18.185960 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:18.186065 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:18.186131 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:18.227220 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0920 05:49:18.381493 9338 executor.go:111] Tasks: 0 done / 120 total; 47 can run I0920 05:49:18.383959 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:18.384031 9338 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0920 05:49:18.384135 9338 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0920 05:49:18.384072 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:18.384168 9338 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0920 05:49:18.384153 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:18.384265 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecilium.example.com" },{ Key: "KubernetesCluster", Value: "privatecilium.example.com" },{ Key: "kubernetes.io/cluster/privatecilium.example.com", Value: "owned" }] }] } W0920 05:49:18.384819 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0920 05:49:18.385781 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0920 05:49:18.386332 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:18.389672 9338 executor.go:111] Tasks: 47 done / 120 total; 28 can run I0920 05:49:18.395705 9338 executor.go:111] Tasks: 75 done / 120 total; 36 can run I0920 05:49:18.396018 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:18.396180 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:18.396406 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:18.396626 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:18.397250 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:18.399225 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:18.399385 9338 executor.go:111] Tasks: 111 done / 120 total; 5 can run I0920 05:49:18.399802 9338 executor.go:111] Tasks: 116 done / 120 total; 4 can run I0920 05:49:18.400075 9338 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0920 05:49:18.404449 9338 target.go:168] Cloudformation output is in /tmp/test2408922992/out --- PASS: TestPrivateCilium (0.59s) === RUN TestPrivateCilium2 I0920 05:49:18.420738 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:18.420970 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:18.421024 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:18.421118 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:18.421172 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:18.421217 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:18.421247 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:18.421286 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:18.421314 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:18.450581 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0920 05:49:18.454673 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:18.454764 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:18.454835 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:18.493354 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:19.130150 9338 executor.go:111] Tasks: 0 done / 124 total; 51 can run W0920 05:49:19.130454 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:19.130608 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:19.130653 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:19.130835 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:19.130888 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:19.131037 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:19.131393 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:19.135703 9338 executor.go:111] Tasks: 51 done / 124 total; 28 can run I0920 05:49:19.145582 9338 executor.go:111] Tasks: 79 done / 124 total; 36 can run I0920 05:49:19.145967 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:19.146230 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:19.146360 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:19.146609 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:19.148331 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:19.149881 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:19.150031 9338 executor.go:111] Tasks: 115 done / 124 total; 5 can run I0920 05:49:19.150608 9338 executor.go:111] Tasks: 120 done / 124 total; 4 can run I0920 05:49:19.150887 9338 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0920 05:49:19.227623 9338 target.go:112] Terraform output is in /tmp/test979052792/out I0920 05:49:19.235299 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:19.235521 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:19.235587 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:19.235642 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:19.235694 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:19.235741 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:19.235770 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:19.235811 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:19.235843 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:19.268353 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0920 05:49:19.272957 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:19.273061 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:19.273130 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:19.311257 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0920 05:49:19.961694 9338 executor.go:111] Tasks: 0 done / 124 total; 51 can run W0920 05:49:19.962172 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0920 05:49:19.962215 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0920 05:49:19.962546 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:19.962764 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:19.962878 9338 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0920 05:49:19.962909 9338 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0920 05:49:19.962917 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:19.963026 9338 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0920 05:49:19.962924 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:19.963089 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecilium.example.com" },{ Key: "KubernetesCluster", Value: "privatecilium.example.com" },{ Key: "kubernetes.io/cluster/privatecilium.example.com", Value: "owned" }] }] } I0920 05:49:19.966794 9338 executor.go:111] Tasks: 51 done / 124 total; 28 can run I0920 05:49:19.977461 9338 executor.go:111] Tasks: 79 done / 124 total; 36 can run I0920 05:49:19.978036 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:19.978285 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:19.978558 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:19.978725 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:19.979503 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:19.980692 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:19.980835 9338 executor.go:111] Tasks: 115 done / 124 total; 5 can run I0920 05:49:19.981214 9338 executor.go:111] Tasks: 120 done / 124 total; 4 can run I0920 05:49:19.981540 9338 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0920 05:49:19.985576 9338 target.go:168] Cloudformation output is in /tmp/test4193234005/out --- PASS: TestPrivateCilium2 (1.58s) === RUN TestPrivateCiliumAdvanced I0920 05:49:19.998252 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:19.998488 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:19.998543 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:19.998637 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:19.998691 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:19.998734 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:19.998762 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:19.998810 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:19.998839 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:20.036351 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:20.040969 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:20.041065 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:20.041134 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:20.080405 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:20.236202 9338 executor.go:111] Tasks: 0 done / 126 total; 53 can run I0920 05:49:20.236454 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:20.236809 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:20.236875 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:20.239888 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:20.239931 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:20.240065 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:20.240252 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:20.244784 9338 executor.go:111] Tasks: 53 done / 126 total; 28 can run I0920 05:49:20.251331 9338 executor.go:111] Tasks: 81 done / 126 total; 36 can run I0920 05:49:20.251844 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:20.251993 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:20.252218 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:20.252447 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:20.253128 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:20.254810 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:20.254940 9338 executor.go:111] Tasks: 117 done / 126 total; 5 can run I0920 05:49:20.255452 9338 executor.go:111] Tasks: 122 done / 126 total; 4 can run I0920 05:49:20.255746 9338 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0920 05:49:20.335175 9338 target.go:112] Terraform output is in /tmp/test805085744/out I0920 05:49:20.339745 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:20.339957 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:20.340041 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:20.340095 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:20.340150 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:20.340194 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:20.340224 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:20.340261 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:20.340290 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:20.375392 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:20.379884 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:20.380004 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:20.380075 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:20.418524 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0920 05:49:20.568023 9338 executor.go:111] Tasks: 0 done / 126 total; 53 can run I0920 05:49:20.568707 9338 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0920 05:49:20.568746 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0920 05:49:20.568847 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:20.568863 9338 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0920 05:49:20.568940 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0920 05:49:20.568977 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:20.569302 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:20.569490 9338 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0920 05:49:20.569521 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:20.569662 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privateciliumadvanced.example.com" },{ Key: "KubernetesCluster", Value: "privateciliumadvanced.example.com" },{ Key: "kubernetes.io/cluster/privateciliumadvanced.example.com", Value: "owned" }] }] } I0920 05:49:20.571932 9338 executor.go:111] Tasks: 53 done / 126 total; 28 can run I0920 05:49:20.578549 9338 executor.go:111] Tasks: 81 done / 126 total; 36 can run I0920 05:49:20.579735 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:20.580258 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:20.581026 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:20.581526 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:20.584917 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:20.586109 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:20.586269 9338 executor.go:111] Tasks: 117 done / 126 total; 5 can run I0920 05:49:20.586688 9338 executor.go:111] Tasks: 122 done / 126 total; 4 can run I0920 05:49:20.587003 9338 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0920 05:49:20.591351 9338 target.go:168] Cloudformation output is in /tmp/test2970159903/out --- PASS: TestPrivateCiliumAdvanced (0.61s) === RUN TestPrivateCanal I0920 05:49:20.605278 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:20.605511 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:20.605562 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:20.605664 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:20.605719 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:20.605762 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:20.605792 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:20.605835 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:20.605862 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:20.632736 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0920 05:49:20.635845 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:20.635920 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:20.635973 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:20.674317 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:21.165191 9338 executor.go:111] Tasks: 0 done / 123 total; 50 can run I0920 05:49:21.166094 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecanal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:21.166199 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:21.166246 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:21.166284 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0920 05:49:21.166394 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:21.166404 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:21.166444 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:21.168083 9338 executor.go:111] Tasks: 50 done / 123 total; 28 can run I0920 05:49:21.177621 9338 executor.go:111] Tasks: 78 done / 123 total; 36 can run I0920 05:49:21.178305 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:21.178448 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:21.178733 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:21.178915 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:21.179725 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:21.180949 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:21.181074 9338 executor.go:111] Tasks: 114 done / 123 total; 5 can run I0920 05:49:21.181571 9338 executor.go:111] Tasks: 119 done / 123 total; 4 can run I0920 05:49:21.181877 9338 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0920 05:49:21.257048 9338 target.go:112] Terraform output is in /tmp/test1972461957/out --- PASS: TestPrivateCanal (0.66s) === RUN TestPrivateKopeio I0920 05:49:21.262579 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:21.262820 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:21.262876 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:21.262983 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:21.263040 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:21.263085 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:21.263113 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:21.263153 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:21.263184 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:21.296135 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:21.299911 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:21.300037 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:21.300111 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:21.339822 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:21.454404 9338 executor.go:111] Tasks: 0 done / 126 total; 47 can run I0920 05:49:21.454682 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:21.454710 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:21.454817 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:21.454851 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:21.455094 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:21.455128 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:21.455490 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:21.457845 9338 executor.go:111] Tasks: 47 done / 126 total; 31 can run I0920 05:49:21.463149 9338 executor.go:111] Tasks: 78 done / 126 total; 39 can run I0920 05:49:21.463382 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:21.463515 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:21.464022 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:21.464145 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:21.465253 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:21.465897 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:21.466018 9338 executor.go:111] Tasks: 117 done / 126 total; 6 can run I0920 05:49:21.466532 9338 executor.go:111] Tasks: 123 done / 126 total; 3 can run I0920 05:49:21.466881 9338 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0920 05:49:21.547130 9338 target.go:112] Terraform output is in /tmp/test902392092/out --- PASS: TestPrivateKopeio (0.29s) === RUN TestUnmanaged I0920 05:49:21.551521 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:21.551761 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:21.551822 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:21.551876 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:21.551930 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:21.552030 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:21.552062 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:21.552102 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:21.552131 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:21.582348 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:21.585837 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:21.585926 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:21.585986 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:21.625029 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:21.728075 9338 executor.go:111] Tasks: 0 done / 108 total; 45 can run I0920 05:49:21.728429 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:21.728441 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.unmanaged.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:21.728476 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:21.728604 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:21.728689 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:21.729056 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:21.729309 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:21.731480 9338 executor.go:111] Tasks: 45 done / 108 total; 25 can run I0920 05:49:21.741156 9338 executor.go:111] Tasks: 70 done / 108 total; 31 can run I0920 05:49:21.741499 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:21.741671 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:21.741860 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:21.742158 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:21.743599 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:21.744635 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:21.744770 9338 executor.go:111] Tasks: 101 done / 108 total; 4 can run I0920 05:49:21.745315 9338 executor.go:111] Tasks: 105 done / 108 total; 3 can run I0920 05:49:21.745745 9338 executor.go:111] Tasks: 108 done / 108 total; 0 can run I0920 05:49:21.815541 9338 target.go:112] Terraform output is in /tmp/test2753889861/out --- PASS: TestUnmanaged (0.27s) === RUN TestPrivateSharedSubnet I0920 05:49:21.819809 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:21.820469 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:21.820535 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:21.820599 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:21.820670 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:21.820780 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:21.820814 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:21.820866 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:21.820899 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:21.852938 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:21.857690 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:21.857820 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:21.857915 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:21.898093 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:22.001972 9338 executor.go:111] Tasks: 0 done / 107 total; 45 can run I0920 05:49:22.002594 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:22.002681 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:22.002892 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:22.002913 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:22.003028 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:22.003060 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:22.003138 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0920 05:49:22.005016 9338 executor.go:111] Tasks: 45 done / 107 total; 24 can run I0920 05:49:22.005711 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0920 05:49:22.010408 9338 executor.go:111] Tasks: 69 done / 107 total; 31 can run I0920 05:49:22.010658 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:22.010960 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:22.011142 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:22.011240 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:22.012312 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:22.013152 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:22.013301 9338 executor.go:111] Tasks: 100 done / 107 total; 4 can run I0920 05:49:22.013851 9338 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0920 05:49:22.014115 9338 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0920 05:49:22.084250 9338 target.go:112] Terraform output is in /tmp/test1058750026/out --- PASS: TestPrivateSharedSubnet (0.27s) === RUN TestPrivateSharedIP I0920 05:49:22.091706 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:22.091958 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:22.092020 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:22.092125 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:22.092180 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:22.092231 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:22.092260 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:22.092300 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:22.092332 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:22.120674 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:22.124379 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:22.124480 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:22.124548 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:22.163406 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:22.266226 9338 executor.go:111] Tasks: 0 done / 116 total; 45 can run I0920 05:49:22.266483 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:22.266678 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:22.267019 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:22.267056 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:22.267140 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:22.267253 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:22.267357 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:22.269045 9338 executor.go:111] Tasks: 45 done / 116 total; 26 can run I0920 05:49:22.269518 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0920 05:49:22.274429 9338 executor.go:111] Tasks: 71 done / 116 total; 36 can run I0920 05:49:22.275009 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:22.275142 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:22.275371 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:22.275535 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:22.276148 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:22.277557 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:22.277687 9338 executor.go:111] Tasks: 107 done / 116 total; 5 can run I0920 05:49:22.279030 9338 executor.go:111] Tasks: 112 done / 116 total; 4 can run I0920 05:49:22.279440 9338 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0920 05:49:22.357382 9338 target.go:112] Terraform output is in /tmp/test1756522200/out I0920 05:49:22.361527 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:22.361737 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:22.361798 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:22.361850 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:22.361904 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:22.361951 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:22.361980 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:22.362018 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:22.362050 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:22.390081 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:22.393715 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:22.393806 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:22.393867 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:22.432597 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0920 05:49:22.534623 9338 executor.go:111] Tasks: 0 done / 116 total; 45 can run I0920 05:49:22.535045 9338 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0920 05:49:22.535094 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:22.535238 9338 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0920 05:49:22.535305 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0920 05:49:22.535657 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0920 05:49:22.535720 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:22.535717 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:22.535905 9338 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0920 05:49:22.535933 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:22.536094 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0920 05:49:22.537638 9338 executor.go:111] Tasks: 45 done / 116 total; 26 can run I0920 05:49:22.538265 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0920 05:49:22.545446 9338 executor.go:111] Tasks: 71 done / 116 total; 36 can run I0920 05:49:22.546820 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:22.547043 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:22.547274 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:22.547477 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:22.548172 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:22.549420 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:22.549573 9338 executor.go:111] Tasks: 107 done / 116 total; 5 can run I0920 05:49:22.549936 9338 executor.go:111] Tasks: 112 done / 116 total; 4 can run I0920 05:49:22.550246 9338 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0920 05:49:22.554648 9338 target.go:168] Cloudformation output is in /tmp/test2135626990/out --- PASS: TestPrivateSharedIP (0.48s) === RUN TestPrivateDns1 I0920 05:49:22.568273 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:22.568506 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:22.568556 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:22.568606 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:22.568654 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:22.568738 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:22.568768 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:22.568806 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:22.568836 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:22.600701 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:22.600783 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:22.600839 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:22.756456 9338 executor.go:111] Tasks: 0 done / 120 total; 46 can run W0920 05:49:22.756734 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:22.756966 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:22.757258 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:22.757671 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns1.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:22.760200 9338 executor.go:111] Tasks: 46 done / 120 total; 24 can run I0920 05:49:22.760758 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "internal.example.com" I0920 05:49:22.760798 9338 zones.go:40] GetHostedZone { Id: "/hostedzone/Z2AFAKE1ZON3NO" } I0920 05:49:22.760886 9338 dnszone.go:249] Existing zone "internal.example.com." found; will configure TF to reuse I0920 05:49:22.760903 9338 dnszone.go:272] No association between VPC "privatedns1.example.com" and zone "internal.example.com."; adding I0920 05:49:22.760986 9338 executor.go:111] Tasks: 70 done / 120 total; 35 can run I0920 05:49:22.761272 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:22.761470 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:22.764650 9338 executor.go:111] Tasks: 105 done / 120 total; 5 can run W0920 05:49:22.764749 9338 api.go:53] PageSize not implemented W0920 05:49:22.764968 9338 api.go:53] PageSize not implemented I0920 05:49:22.769724 9338 executor.go:111] Tasks: 110 done / 120 total; 6 can run I0920 05:49:22.770030 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:22.770251 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:22.771187 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:22.772494 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:22.772626 9338 executor.go:111] Tasks: 116 done / 120 total; 2 can run I0920 05:49:22.772936 9338 executor.go:111] Tasks: 118 done / 120 total; 2 can run I0920 05:49:22.773096 9338 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0920 05:49:22.857964 9338 target.go:112] Terraform output is in /tmp/test2547802334/out --- PASS: TestPrivateDns1 (0.29s) === RUN TestPrivateDns2 I0920 05:49:22.862618 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:22.862867 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:22.862930 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:22.862985 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:22.863042 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:22.863137 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:22.863171 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:22.863212 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:22.863243 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:22.895085 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:22.895186 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:22.895253 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:23.029176 9338 executor.go:111] Tasks: 0 done / 116 total; 44 can run W0920 05:49:23.029644 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:23.030115 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:23.030121 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:23.032082 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns2.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:23.037019 9338 executor.go:111] Tasks: 44 done / 116 total; 22 can run I0920 05:49:23.037406 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "private.example.com" I0920 05:49:23.037414 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0920 05:49:23.037451 9338 zones.go:40] GetHostedZone { Id: "/hostedzone/Z3AFAKE1ZOMORE" } I0920 05:49:23.037533 9338 dnszone.go:249] Existing zone "private.example.com." found; will configure TF to reuse I0920 05:49:23.037553 9338 dnszone.go:263] VPC "vpc-12345678" already associated with zone "private.example.com." I0920 05:49:23.037671 9338 executor.go:111] Tasks: 66 done / 116 total; 35 can run I0920 05:49:23.038244 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:23.038430 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:23.041225 9338 executor.go:111] Tasks: 101 done / 116 total; 5 can run W0920 05:49:23.041445 9338 api.go:53] PageSize not implemented W0920 05:49:23.041564 9338 api.go:53] PageSize not implemented I0920 05:49:23.047226 9338 executor.go:111] Tasks: 106 done / 116 total; 6 can run I0920 05:49:23.047392 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:23.047607 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:23.048481 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:23.049999 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:23.050134 9338 executor.go:111] Tasks: 112 done / 116 total; 2 can run I0920 05:49:23.050509 9338 executor.go:111] Tasks: 114 done / 116 total; 2 can run I0920 05:49:23.050708 9338 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0920 05:49:23.124841 9338 target.go:112] Terraform output is in /tmp/test2461930833/out --- PASS: TestPrivateDns2 (0.27s) === RUN TestDiscoveryFeatureGate I0920 05:49:23.129070 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:23.129297 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:23.129356 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:23.129457 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:23.129508 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:23.129551 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:23.129581 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:23.129617 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:23.129648 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:23.155287 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0920 05:49:23.159065 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:23.159165 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:23.197431 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:23.402078 9338 executor.go:111] Tasks: 0 done / 99 total; 51 can run W0920 05:49:23.402586 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:23.403280 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:23.404073 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:23.407511 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:23.407564 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:23.407699 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0920 05:49:23.409231 9338 executor.go:111] Tasks: 51 done / 99 total; 23 can run I0920 05:49:23.415091 9338 executor.go:111] Tasks: 74 done / 99 total; 21 can run I0920 05:49:23.415625 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:23.415880 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:23.417422 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:23.418898 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:23.419026 9338 executor.go:111] Tasks: 95 done / 99 total; 2 can run I0920 05:49:23.419365 9338 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0920 05:49:23.419570 9338 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0920 05:49:23.481610 9338 target.go:112] Terraform output is in /tmp/test4144098456/out --- PASS: TestDiscoveryFeatureGate (0.36s) === RUN TestVFSServiceAccountIssuerDiscovery I0920 05:49:23.487094 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:23.487329 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:23.487393 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:23.487449 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:23.487504 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:23.487600 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:23.487630 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:23.487670 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:23.487701 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:23.512440 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:23.515815 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:23.515914 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:23.554084 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:23.652330 9338 executor.go:111] Tasks: 0 done / 90 total; 45 can run W0920 05:49:23.652829 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:23.653193 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:23.653194 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:23.653242 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:23.653411 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:23.653488 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:23.657950 9338 executor.go:111] Tasks: 45 done / 90 total; 20 can run I0920 05:49:23.665122 9338 executor.go:111] Tasks: 65 done / 90 total; 21 can run I0920 05:49:23.665387 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:23.665782 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:23.666559 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:23.668584 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:23.668704 9338 executor.go:111] Tasks: 86 done / 90 total; 2 can run I0920 05:49:23.669021 9338 executor.go:111] Tasks: 88 done / 90 total; 2 can run I0920 05:49:23.669205 9338 executor.go:111] Tasks: 90 done / 90 total; 0 can run I0920 05:49:23.723916 9338 target.go:112] Terraform output is in /tmp/test3238338954/out --- PASS: TestVFSServiceAccountIssuerDiscovery (0.24s) === RUN TestAWSLBController I0920 05:49:23.728217 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:23.728446 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:23.728503 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:23.728551 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:23.728605 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:23.728692 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:23.728722 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:23.728758 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:23.728788 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:23.754465 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:23.757705 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:23.757793 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:23.796322 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:24.372382 9338 executor.go:111] Tasks: 0 done / 96 total; 49 can run W0920 05:49:24.372839 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:24.373291 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:24.373486 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:24.373688 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:24.373721 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:24.373856 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0920 05:49:24.376238 9338 executor.go:111] Tasks: 49 done / 96 total; 22 can run I0920 05:49:24.386430 9338 executor.go:111] Tasks: 71 done / 96 total; 21 can run I0920 05:49:24.386886 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:24.387109 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:24.388296 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:24.389531 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:24.389689 9338 executor.go:111] Tasks: 92 done / 96 total; 2 can run I0920 05:49:24.390045 9338 executor.go:111] Tasks: 94 done / 96 total; 2 can run I0920 05:49:24.390251 9338 executor.go:111] Tasks: 96 done / 96 total; 0 can run I0920 05:49:24.445999 9338 target.go:112] Terraform output is in /tmp/test3957395719/out --- PASS: TestAWSLBController (0.73s) === RUN TestManyAddons I0920 05:49:24.457432 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:24.457746 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:24.457808 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:24.457880 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:24.457951 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:24.458093 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:24.458145 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:24.458191 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:24.458230 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:24.487521 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:24.492113 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:24.492218 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:24.531153 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:25.270284 9338 executor.go:111] Tasks: 0 done / 95 total; 50 can run W0920 05:49:25.270904 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:25.271311 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:25.271530 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:25.271543 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:25.271573 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:25.271677 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0920 05:49:25.274494 9338 executor.go:111] Tasks: 50 done / 95 total; 19 can run I0920 05:49:25.284153 9338 executor.go:111] Tasks: 69 done / 95 total; 22 can run I0920 05:49:25.284676 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:25.284878 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:25.286218 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:25.287889 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:25.288016 9338 executor.go:111] Tasks: 91 done / 95 total; 2 can run I0920 05:49:25.288344 9338 executor.go:111] Tasks: 93 done / 95 total; 2 can run I0920 05:49:25.288625 9338 executor.go:111] Tasks: 95 done / 95 total; 0 can run I0920 05:49:25.340264 9338 target.go:112] Terraform output is in /tmp/test2067121726/out --- PASS: TestManyAddons (0.90s) === RUN TestManyAddonsCCMIRSA I0920 05:49:25.353024 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:25.353335 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:25.353399 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:25.353502 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:25.353556 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:25.353601 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:25.353634 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:25.353673 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:25.353702 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:25.381524 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:25.385635 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:25.385733 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:25.424322 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:26.253196 9338 executor.go:111] Tasks: 0 done / 111 total; 59 can run W0920 05:49:26.253668 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:26.253817 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:26.253986 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:26.254024 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:26.254119 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0920 05:49:26.254544 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:26.261583 9338 executor.go:111] Tasks: 59 done / 111 total; 26 can run I0920 05:49:26.267976 9338 executor.go:111] Tasks: 85 done / 111 total; 22 can run I0920 05:49:26.268311 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:26.268595 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:26.269398 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:26.271222 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:26.271351 9338 executor.go:111] Tasks: 107 done / 111 total; 2 can run I0920 05:49:26.271737 9338 executor.go:111] Tasks: 109 done / 111 total; 2 can run I0920 05:49:26.271939 9338 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0920 05:49:26.342896 9338 target.go:112] Terraform output is in /tmp/test247050979/out --- PASS: TestManyAddonsCCMIRSA (1.00s) === RUN TestManyAddonsCCMIRSA23 I0920 05:49:26.352259 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:26.352498 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:26.352562 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:26.352676 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:26.352731 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:26.352778 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:26.352806 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:26.352843 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:26.352873 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:26.379817 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I0920 05:49:26.383173 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:26.383257 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:26.422339 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:27.292686 9338 executor.go:111] Tasks: 0 done / 112 total; 60 can run W0920 05:49:27.294821 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:27.295323 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:27.295713 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:27.295753 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:27.295944 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:27.293409 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:27.300728 9338 executor.go:111] Tasks: 60 done / 112 total; 26 can run I0920 05:49:27.311100 9338 executor.go:111] Tasks: 86 done / 112 total; 22 can run I0920 05:49:27.311393 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:27.311779 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:27.312723 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:27.314738 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:27.314879 9338 executor.go:111] Tasks: 108 done / 112 total; 2 can run I0920 05:49:27.315254 9338 executor.go:111] Tasks: 110 done / 112 total; 2 can run I0920 05:49:27.315458 9338 executor.go:111] Tasks: 112 done / 112 total; 0 can run I0920 05:49:27.384674 9338 target.go:112] Terraform output is in /tmp/test2138515061/out --- PASS: TestManyAddonsCCMIRSA23 (1.04s) === RUN TestManyAddonsCCMIRSA24 I0920 05:49:27.394834 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:27.395063 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:27.395128 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:27.395240 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:27.395293 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:27.395341 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:27.395369 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:27.395407 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:27.395436 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:27.422952 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0920 05:49:27.426864 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:27.426956 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:27.467430 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:28.333524 9338 executor.go:111] Tasks: 0 done / 112 total; 60 can run I0920 05:49:28.334249 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:28.334295 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:28.334390 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:28.334823 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:28.336216 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:28.336488 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:28.342270 9338 executor.go:111] Tasks: 60 done / 112 total; 26 can run I0920 05:49:28.347630 9338 executor.go:111] Tasks: 86 done / 112 total; 22 can run I0920 05:49:28.348017 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:28.348227 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:28.349450 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:28.350636 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:28.350768 9338 executor.go:111] Tasks: 108 done / 112 total; 2 can run I0920 05:49:28.351098 9338 executor.go:111] Tasks: 110 done / 112 total; 2 can run I0920 05:49:28.351322 9338 executor.go:111] Tasks: 112 done / 112 total; 0 can run I0920 05:49:28.421873 9338 target.go:112] Terraform output is in /tmp/test1291547779/out --- PASS: TestManyAddonsCCMIRSA24 (1.04s) === RUN TestManyAddonsCCMIRSA25 I0920 05:49:28.430973 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:28.431238 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:28.431301 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:28.431405 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:28.431460 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:28.431509 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:28.431539 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:28.431579 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:28.431608 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:28.459163 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0920 05:49:28.462794 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:28.462884 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:28.504603 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:29.409386 9338 executor.go:111] Tasks: 0 done / 112 total; 60 can run W0920 05:49:29.410028 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:29.410142 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:29.410225 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:29.410189 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:29.410338 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:29.410278 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:29.419703 9338 executor.go:111] Tasks: 60 done / 112 total; 26 can run I0920 05:49:29.425687 9338 executor.go:111] Tasks: 86 done / 112 total; 22 can run I0920 05:49:29.426141 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:29.426377 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:29.427622 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:29.429145 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:29.429291 9338 executor.go:111] Tasks: 108 done / 112 total; 2 can run I0920 05:49:29.429616 9338 executor.go:111] Tasks: 110 done / 112 total; 2 can run I0920 05:49:29.429816 9338 executor.go:111] Tasks: 112 done / 112 total; 0 can run I0920 05:49:29.500867 9338 target.go:112] Terraform output is in /tmp/test1526481086/out --- PASS: TestManyAddonsCCMIRSA25 (1.08s) === RUN TestManyAddonsCCMIRSA26 I0920 05:49:29.510739 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:29.511009 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:29.511070 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:29.511132 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:29.511201 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:29.511304 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:29.511335 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:29.511383 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:29.511415 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:29.538354 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* This version of kubernetes is not yet supported; upgrading kops is required (you can bypass this check by exporting KOPS_RUN_TOO_NEW_VERSION) ********************************************************************************* I0920 05:49:29.542027 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:29.542115 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:29.587335 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:30.454573 9338 executor.go:111] Tasks: 0 done / 111 total; 59 can run I0920 05:49:30.454871 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:30.454899 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:30.454993 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:30.455477 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:30.455588 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:30.455578 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:30.465906 9338 executor.go:111] Tasks: 59 done / 111 total; 26 can run I0920 05:49:30.472007 9338 executor.go:111] Tasks: 85 done / 111 total; 22 can run I0920 05:49:30.472491 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:30.472703 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:30.473738 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:30.475563 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:30.475700 9338 executor.go:111] Tasks: 107 done / 111 total; 2 can run I0920 05:49:30.476082 9338 executor.go:111] Tasks: 109 done / 111 total; 2 can run I0920 05:49:30.476304 9338 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0920 05:49:30.544156 9338 target.go:112] Terraform output is in /tmp/test2000995212/out --- PASS: TestManyAddonsCCMIRSA26 (1.05s) === RUN TestCCM I0920 05:49:30.557023 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:30.557288 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:30.557354 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:30.557410 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:30.557465 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:30.557556 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:30.557586 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:30.557624 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:30.557652 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:30.586255 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:30.590236 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:30.590338 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:30.628651 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:31.430159 9338 executor.go:111] Tasks: 0 done / 96 total; 51 can run I0920 05:49:31.430739 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:31.430942 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:31.430985 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:31.431118 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:31.431120 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:31.431218 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:31.437155 9338 executor.go:111] Tasks: 51 done / 96 total; 19 can run I0920 05:49:31.444452 9338 executor.go:111] Tasks: 70 done / 96 total; 22 can run I0920 05:49:31.444912 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:31.445532 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:31.446860 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:31.448125 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:31.448321 9338 executor.go:111] Tasks: 92 done / 96 total; 2 can run I0920 05:49:31.448703 9338 executor.go:111] Tasks: 94 done / 96 total; 2 can run I0920 05:49:31.448885 9338 executor.go:111] Tasks: 96 done / 96 total; 0 can run I0920 05:49:31.502366 9338 target.go:112] Terraform output is in /tmp/test978439869/out --- PASS: TestCCM (0.96s) === RUN TestExternalDNS I0920 05:49:31.513752 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:31.514006 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:31.514067 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:31.514182 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:31.514241 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:31.514287 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:31.514318 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:31.514355 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:31.514384 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:31.539702 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:31.542997 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:31.543095 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:31.583188 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:31.681862 9338 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0920 05:49:31.682329 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:31.682374 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:31.682512 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:31.682540 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:31.682481 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:31.682752 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:31.685318 9338 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0920 05:49:31.690380 9338 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0920 05:49:31.690805 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:31.691082 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:31.691965 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:31.693250 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:31.693391 9338 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0920 05:49:31.693691 9338 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0920 05:49:31.693946 9338 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0920 05:49:31.746797 9338 target.go:112] Terraform output is in /tmp/test1126840105/out I0920 05:49:31.750713 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:31.750930 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:31.750991 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:31.751043 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:31.751095 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:31.751143 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:31.751171 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:31.751211 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:31.751239 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:31.775573 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:31.778899 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:31.778994 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:31.816937 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0920 05:49:31.920007 9338 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0920 05:49:31.920520 9338 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0920 05:49:31.920550 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:31.920656 9338 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0920 05:49:31.920520 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:31.920770 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:31.920856 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0920 05:49:31.920956 9338 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0920 05:49:31.920977 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:31.921244 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0920 05:49:31.928527 9338 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0920 05:49:31.934799 9338 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0920 05:49:31.935349 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:31.935596 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:31.936823 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:31.937795 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:31.937943 9338 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0920 05:49:31.938191 9338 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0920 05:49:31.938362 9338 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0920 05:49:31.941350 9338 target.go:168] Cloudformation output is in /tmp/test2599301571/out --- PASS: TestExternalDNS (0.44s) === RUN TestExternalDNSIRSA I0920 05:49:31.951561 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:31.951790 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:31.951846 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:31.951892 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:31.951943 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:31.952022 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:31.952057 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:31.952107 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:31.952155 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:31.982095 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:31.985429 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:31.985525 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:32.023242 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:32.141848 9338 executor.go:111] Tasks: 0 done / 92 total; 46 can run W0920 05:49:32.142377 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:32.142856 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:32.142916 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:32.143156 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:32.143192 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:32.143301 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0920 05:49:32.145697 9338 executor.go:111] Tasks: 46 done / 92 total; 21 can run I0920 05:49:32.157387 9338 executor.go:111] Tasks: 67 done / 92 total; 21 can run I0920 05:49:32.157606 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:32.157752 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:32.158738 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:32.160380 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:32.160501 9338 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0920 05:49:32.160818 9338 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0920 05:49:32.161008 9338 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0920 05:49:32.217932 9338 target.go:112] Terraform output is in /tmp/test2940425472/out --- PASS: TestExternalDNSIRSA (0.27s) === RUN TestKarpenter I0920 05:49:32.222334 9338 featureflag.go:160] FeatureFlag "Karpenter"=true I0920 05:49:32.222383 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:32.222606 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:32.222660 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:32.222708 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:32.222759 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:32.222850 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:32.222879 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:32.222918 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:32.222950 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:32.253611 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0920 05:49:32.257706 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:32.257789 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:32.257849 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:32.296432 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:32.482454 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:32.482625 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:32.482748 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:32.604509 9338 executor.go:111] Tasks: 0 done / 108 total; 53 can run I0920 05:49:32.604802 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:32.604836 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:32.604937 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:32.605127 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:32.605252 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:32.605435 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:32.608167 9338 executor.go:111] Tasks: 53 done / 108 total; 26 can run I0920 05:49:32.617461 9338 executor.go:111] Tasks: 79 done / 108 total; 25 can run I0920 05:49:32.617748 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:32.617898 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:32.618117 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:32.618336 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:32.618953 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:32.619338 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:32.619490 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:32.620530 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:32.620685 9338 executor.go:111] Tasks: 104 done / 108 total; 2 can run I0920 05:49:32.621168 9338 executor.go:111] Tasks: 106 done / 108 total; 2 can run I0920 05:49:32.621460 9338 executor.go:111] Tasks: 108 done / 108 total; 0 can run I0920 05:49:32.692199 9338 target.go:112] Terraform output is in /tmp/test2090738027/out I0920 05:49:32.698962 9338 featureflag.go:160] FeatureFlag "Karpenter"=false --- PASS: TestKarpenter (0.48s) === RUN TestSharedSubnet I0920 05:49:32.699105 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:32.699339 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:32.699404 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:32.699457 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:32.699512 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:32.699609 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:32.699639 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:32.699678 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:32.699711 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:32.724213 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:32.727305 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:32.727402 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:32.767209 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:32.870834 9338 executor.go:111] Tasks: 0 done / 80 total; 42 can run W0920 05:49:32.871170 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:32.871641 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:32.871679 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:32.872639 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0920 05:49:32.871475 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:32.873627 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:32.879059 9338 executor.go:111] Tasks: 42 done / 80 total; 16 can run I0920 05:49:32.879536 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0920 05:49:32.884582 9338 executor.go:111] Tasks: 58 done / 80 total; 18 can run I0920 05:49:32.884920 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:32.885185 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:32.886309 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:32.887990 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:32.888112 9338 executor.go:111] Tasks: 76 done / 80 total; 2 can run I0920 05:49:32.888472 9338 executor.go:111] Tasks: 78 done / 80 total; 2 can run I0920 05:49:32.888668 9338 executor.go:111] Tasks: 80 done / 80 total; 0 can run I0920 05:49:32.933240 9338 target.go:112] Terraform output is in /tmp/test3524368820/out --- PASS: TestSharedSubnet (0.24s) === RUN TestSharedVPC I0920 05:49:32.937517 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:32.937753 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:32.937811 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:32.937865 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:32.937918 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:32.938019 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:32.938050 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:32.938087 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:32.938114 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:32.966974 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:32.970342 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:32.970451 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:33.008580 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:33.109758 9338 executor.go:111] Tasks: 0 done / 84 total; 42 can run W0920 05:49:33.110237 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:33.110299 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:33.110395 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:33.110435 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:33.110576 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0920 05:49:33.110687 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:33.113178 9338 executor.go:111] Tasks: 42 done / 84 total; 17 can run I0920 05:49:33.113451 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0920 05:49:33.118704 9338 executor.go:111] Tasks: 59 done / 84 total; 21 can run I0920 05:49:33.119073 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:33.119299 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:33.120257 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:33.121544 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:33.121665 9338 executor.go:111] Tasks: 80 done / 84 total; 2 can run I0920 05:49:33.121981 9338 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0920 05:49:33.122172 9338 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0920 05:49:33.173149 9338 target.go:112] Terraform output is in /tmp/test3066417453/out --- PASS: TestSharedVPC (0.24s) === RUN TestExistingIAM I0920 05:49:33.177225 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:33.177485 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:33.177549 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:33.177604 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:33.177661 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:33.177761 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:33.177792 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:33.177834 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:33.177864 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:33.211214 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:33.214894 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:33.214980 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:33.215036 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:33.215094 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:33.260213 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:33.368720 9338 executor.go:111] Tasks: 0 done / 101 total; 51 can run W0920 05:49:33.369322 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:33.369298 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existing-iam.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:33.369412 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:33.369525 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:33.371468 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:33.369704 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:33.373109 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:33.373357 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0920 05:49:33.376368 9338 executor.go:111] Tasks: 51 done / 101 total; 15 can run I0920 05:49:33.381823 9338 executor.go:111] Tasks: 66 done / 101 total; 27 can run I0920 05:49:33.382172 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:33.382329 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:33.382547 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:33.382814 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:33.383682 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:33.384460 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:33.384928 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:33.385191 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:33.385312 9338 executor.go:111] Tasks: 93 done / 101 total; 4 can run I0920 05:49:33.385689 9338 executor.go:111] Tasks: 97 done / 101 total; 4 can run I0920 05:49:33.385954 9338 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0920 05:49:33.461902 9338 target.go:112] Terraform output is in /tmp/test3444995791/out --- PASS: TestExistingIAM (0.29s) === RUN TestPhaseNetwork I0920 05:49:33.466522 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:33.466776 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:33.466836 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:33.466891 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:33.466949 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:33.466994 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:33.467022 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:33.467061 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:33.467096 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:33.495699 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:33.499198 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:33.499288 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:33.499348 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:33.537017 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:33.641693 9338 executor.go:111] Tasks: 0 done / 119 total; 46 can run I0920 05:49:33.642072 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.lifecyclephases.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:33.642197 9338 executor.go:111] Tasks: 46 done / 119 total; 28 can run I0920 05:49:33.642676 9338 executor.go:111] Tasks: 74 done / 119 total; 36 can run I0920 05:49:33.643116 9338 executor.go:111] Tasks: 110 done / 119 total; 5 can run I0920 05:49:33.643431 9338 executor.go:111] Tasks: 115 done / 119 total; 4 can run I0920 05:49:33.643670 9338 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0920 05:49:33.656141 9338 target.go:112] Terraform output is in /tmp/test3763513365/out --- PASS: TestPhaseNetwork (0.19s) === RUN TestExternalLoadBalancer I0920 05:49:33.656957 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:33.657188 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:33.657241 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:33.657348 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:33.657403 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:33.657450 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:33.657480 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:33.657518 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:33.657546 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:33.684017 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:33.687362 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:33.687472 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:33.725558 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:33.827355 9338 executor.go:111] Tasks: 0 done / 94 total; 50 can run W0920 05:49:33.827697 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:33.828042 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:33.828250 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:33.828286 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:33.828436 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0920 05:49:33.828488 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:33.832061 9338 executor.go:111] Tasks: 50 done / 94 total; 19 can run I0920 05:49:33.840724 9338 executor.go:111] Tasks: 69 done / 94 total; 21 can run I0920 05:49:33.841008 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:33.841370 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:33.842563 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:33.843931 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:33.844067 9338 executor.go:111] Tasks: 90 done / 94 total; 2 can run I0920 05:49:33.844433 9338 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0920 05:49:33.844641 9338 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0920 05:49:33.894374 9338 target.go:112] Terraform output is in /tmp/test1970388394/out I0920 05:49:33.901078 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:33.901342 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:33.901393 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:33.901447 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:33.901503 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:33.901558 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:33.901589 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:33.901627 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:33.901654 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:33.925173 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:33.927459 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:33.927532 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:33.965682 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0920 05:49:34.047410 9338 executor.go:111] Tasks: 0 done / 94 total; 50 can run W0920 05:49:34.047961 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:34.048085 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:34.048241 9338 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0920 05:49:34.048310 9338 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0920 05:49:34.048265 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:34.048371 9338 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0920 05:49:34.048345 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:34.048145 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:34.048538 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0920 05:49:34.050659 9338 executor.go:111] Tasks: 50 done / 94 total; 19 can run I0920 05:49:34.055919 9338 executor.go:111] Tasks: 69 done / 94 total; 21 can run I0920 05:49:34.056303 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.056542 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.057725 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.058740 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.058878 9338 executor.go:111] Tasks: 90 done / 94 total; 2 can run I0920 05:49:34.059144 9338 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0920 05:49:34.059342 9338 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0920 05:49:34.062391 9338 target.go:168] Cloudformation output is in /tmp/test563256167/out --- PASS: TestExternalLoadBalancer (0.42s) === RUN TestPhaseIAM integration_test.go:853: unable to test w/o allowing failed validation --- SKIP: TestPhaseIAM (0.00s) === RUN TestPhaseCluster integration_test.go:861: unable to test w/o allowing failed validation --- SKIP: TestPhaseCluster (0.00s) === RUN TestMixedInstancesASG I0920 05:49:34.077451 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:34.077823 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:34.077927 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:34.078041 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:34.078126 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:34.078232 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:34.078277 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:34.078331 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:34.078370 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:34.113528 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:34.117554 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.117642 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.117699 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.117757 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.117813 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.117864 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.117917 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.156873 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:34.271090 9338 executor.go:111] Tasks: 0 done / 111 total; 53 can run W0920 05:49:34.272015 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:34.272139 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0920 05:49:34.272155 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:34.272176 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:34.272282 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:34.272147 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:34.272320 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:34.272319 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:34.273966 9338 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0920 05:49:34.285445 9338 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0920 05:49:34.285988 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.286249 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.286483 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.286772 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.287731 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.290246 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.290432 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.290617 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.290771 9338 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0920 05:49:34.291217 9338 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0920 05:49:34.291522 9338 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0920 05:49:34.376231 9338 target.go:112] Terraform output is in /tmp/test3540799492/out I0920 05:49:34.381427 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:34.381644 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:34.381710 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:34.381764 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:34.381821 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:34.381871 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:34.381903 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:34.381946 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:34.381977 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:34.414712 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:34.421949 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.422059 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.422123 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.422186 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.422241 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.422294 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.422344 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.460790 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0920 05:49:34.572701 9338 executor.go:111] Tasks: 0 done / 111 total; 53 can run W0920 05:49:34.573249 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:34.573293 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:34.573521 9338 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key W0920 05:49:34.573624 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:34.573554 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:34.573718 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:34.573715 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/mixedinstances.example.com", Value: "owned" },{ Key: "Name", Value: "mixedinstances.example.com" },{ Key: "KubernetesCluster", Value: "mixedinstances.example.com" }] }] } I0920 05:49:34.574031 9338 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0920 05:49:34.574083 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:34.574076 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:34.574201 9338 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0920 05:49:34.576751 9338 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0920 05:49:34.582380 9338 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0920 05:49:34.582763 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.582953 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.583165 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.583440 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.584056 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.586121 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.586943 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.587421 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.587697 9338 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0920 05:49:34.588347 9338 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0920 05:49:34.589418 9338 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0920 05:49:34.595533 9338 target.go:168] Cloudformation output is in /tmp/test3459453712/out --- PASS: TestMixedInstancesASG (0.53s) === RUN TestMixedInstancesSpotASG I0920 05:49:34.612108 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:34.612341 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:34.612394 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:34.612489 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:34.612541 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:34.612581 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:34.612608 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:34.612646 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:34.612673 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:34.643900 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:34.647107 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.647179 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.647227 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.647276 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.647319 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.647358 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.647396 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.685229 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:34.795976 9338 executor.go:111] Tasks: 0 done / 111 total; 53 can run I0920 05:49:34.796245 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:34.796285 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:34.796416 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:34.796736 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:34.796738 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:34.796836 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:34.797171 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:34.797204 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:34.805030 9338 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0920 05:49:34.812079 9338 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0920 05:49:34.812536 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.812808 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.813113 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.813362 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.817802 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.819946 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.820756 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.821196 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.821330 9338 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0920 05:49:34.821798 9338 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0920 05:49:34.822078 9338 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0920 05:49:34.900368 9338 target.go:112] Terraform output is in /tmp/test2857520287/out I0920 05:49:34.905517 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:34.905753 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:34.905812 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:34.905864 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:34.905920 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:34.905970 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:34.905999 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:34.906037 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:34.906069 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:34.943046 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:34.948835 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.948942 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.949006 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.949065 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.949119 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.949169 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.949217 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:34.987292 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0920 05:49:35.100311 9338 executor.go:111] Tasks: 0 done / 111 total; 53 can run I0920 05:49:35.100751 9338 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0920 05:49:35.100871 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:35.100798 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:35.100933 9338 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0920 05:49:35.101291 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0920 05:49:35.101291 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0920 05:49:35.101367 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:35.101380 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:35.101519 9338 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0920 05:49:35.101538 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:35.101642 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "mixedinstances.example.com" },{ Key: "KubernetesCluster", Value: "mixedinstances.example.com" },{ Key: "kubernetes.io/cluster/mixedinstances.example.com", Value: "owned" }] }] } I0920 05:49:35.103974 9338 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0920 05:49:35.112398 9338 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0920 05:49:35.112802 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:35.113050 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:35.113278 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:35.113565 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:35.115833 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:35.117456 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:35.119746 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:35.120878 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:35.121111 9338 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0920 05:49:35.121526 9338 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0920 05:49:35.121800 9338 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0920 05:49:35.126451 9338 target.go:168] Cloudformation output is in /tmp/test913028896/out --- PASS: TestMixedInstancesSpotASG (0.53s) === RUN TestAdditionalObjects I0920 05:49:35.143208 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:35.143444 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:35.143495 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:35.143545 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:35.143595 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:35.143674 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:35.143705 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:35.143743 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:35.143770 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:35.170095 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I0920 05:49:35.173183 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0920 05:49:35.173268 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0920 05:49:35.212312 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:35.375242 9338 executor.go:111] Tasks: 0 done / 92 total; 45 can run I0920 05:49:35.375524 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:35.375557 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:35.375647 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:35.375790 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:35.376028 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:35.376392 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.additionalobjects.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:35.378570 9338 executor.go:111] Tasks: 45 done / 92 total; 19 can run I0920 05:49:35.388391 9338 executor.go:111] Tasks: 64 done / 92 total; 24 can run I0920 05:49:35.388723 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0920 05:49:35.389036 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0920 05:49:35.390331 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0920 05:49:35.391706 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0920 05:49:35.391832 9338 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0920 05:49:35.392138 9338 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0920 05:49:35.392382 9338 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0920 05:49:35.450092 9338 target.go:112] Terraform output is in /tmp/test360031398/out --- PASS: TestAdditionalObjects (0.31s) === RUN TestContainerd I0920 05:49:35.454723 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:35.454964 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:35.455029 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:35.455139 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:35.455195 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:35.455245 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:35.455276 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:35.455314 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:35.455344 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:35.479747 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:35.483045 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:35.483142 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:35.522306 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0920 05:49:35.623546 9338 executor.go:111] Tasks: 0 done / 87 total; 43 can run W0920 05:49:35.623802 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0920 05:49:35.624341 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:35.624350 9338 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0920 05:49:35.624405 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:35.624545 9338 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0920 05:49:35.624534 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:35.624750 9338 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0920 05:49:35.624782 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:35.624921 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "containerd.example.com" },{ Key: "KubernetesCluster", Value: "containerd.example.com" },{ Key: "kubernetes.io/cluster/containerd.example.com", Value: "owned" }] }] } I0920 05:49:35.626794 9338 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0920 05:49:35.632087 9338 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0920 05:49:35.632354 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:35.632645 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:35.633640 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:35.634873 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:35.635010 9338 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0920 05:49:35.635279 9338 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0920 05:49:35.635473 9338 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0920 05:49:35.638520 9338 target.go:168] Cloudformation output is in /tmp/test2185768503/out --- PASS: TestContainerd (0.20s) === RUN TestContainerdCustom I0920 05:49:35.652381 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:35.652741 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:35.652805 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:35.652898 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:35.652954 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:35.652996 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:35.653026 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:35.653064 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:35.653091 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:35.679437 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:35.682975 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:35.683072 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:35.721430 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0920 05:49:35.824948 9338 executor.go:111] Tasks: 0 done / 87 total; 43 can run W0920 05:49:35.825204 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0920 05:49:35.825620 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:35.825751 9338 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0920 05:49:35.825796 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:35.825941 9338 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0920 05:49:35.825976 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:35.826150 9338 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0920 05:49:35.826177 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:35.826309 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "containerd.example.com" },{ Key: "KubernetesCluster", Value: "containerd.example.com" },{ Key: "kubernetes.io/cluster/containerd.example.com", Value: "owned" }] }] } I0920 05:49:35.827751 9338 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0920 05:49:35.833196 9338 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0920 05:49:35.833456 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:35.833853 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:35.834747 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:35.836303 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:35.836444 9338 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0920 05:49:35.836762 9338 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0920 05:49:35.836985 9338 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0920 05:49:35.840001 9338 target.go:168] Cloudformation output is in /tmp/test494960106/out --- PASS: TestContainerdCustom (0.20s) === RUN TestDockerCustom I0920 05:49:35.853153 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:35.853410 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:35.853468 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:35.853566 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:35.853621 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:35.853665 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:35.853722 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:35.853783 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:35.853825 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:35.879969 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:35.883553 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:35.883648 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:35.922400 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0920 05:49:36.027740 9338 executor.go:111] Tasks: 0 done / 87 total; 43 can run W0920 05:49:36.027994 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:36.028357 9338 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0920 05:49:36.028402 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:36.028385 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:36.028469 9338 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0920 05:49:36.028478 9338 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key W0920 05:49:36.028501 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:36.028487 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:36.028542 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "docker.example.com" },{ Key: "KubernetesCluster", Value: "docker.example.com" },{ Key: "kubernetes.io/cluster/docker.example.com", Value: "owned" }] }] } I0920 05:49:36.030551 9338 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0920 05:49:36.035547 9338 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0920 05:49:36.035832 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:36.036013 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:36.037382 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:36.038521 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:36.038674 9338 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0920 05:49:36.038982 9338 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0920 05:49:36.039193 9338 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0920 05:49:36.042275 9338 target.go:168] Cloudformation output is in /tmp/test1758682241/out --- PASS: TestDockerCustom (0.20s) === RUN TestAPIServerNodes I0920 05:49:36.055794 9338 featureflag.go:160] FeatureFlag "APIServerNodes"=true I0920 05:49:36.055843 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:36.056053 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:36.056104 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:36.056199 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:36.056251 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:36.056291 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:36.056319 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:36.056357 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:36.056384 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:36.084227 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.13 ********************************************************************************* I0920 05:49:36.087866 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:36.087959 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:36.088021 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:36.128459 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0920 05:49:36.281407 9338 executor.go:111] Tasks: 0 done / 100 total; 47 can run W0920 05:49:36.281916 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:36.281917 9338 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0920 05:49:36.282038 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:36.281988 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:36.282130 9338 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0920 05:49:36.281931 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:36.282361 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:36.282490 9338 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0920 05:49:36.282507 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:36.282589 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0920 05:49:36.284036 9338 executor.go:111] Tasks: 47 done / 100 total; 24 can run I0920 05:49:36.289516 9338 executor.go:111] Tasks: 71 done / 100 total; 23 can run I0920 05:49:36.289737 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:36.289812 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:36.289996 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:36.290727 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:36.291109 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:36.292271 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:36.292413 9338 executor.go:111] Tasks: 94 done / 100 total; 3 can run I0920 05:49:36.292724 9338 executor.go:111] Tasks: 97 done / 100 total; 3 can run I0920 05:49:36.292976 9338 executor.go:111] Tasks: 100 done / 100 total; 0 can run I0920 05:49:36.296561 9338 target.go:168] Cloudformation output is in /tmp/test1669647014/out I0920 05:49:36.308612 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:36.308831 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:36.308886 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:36.308934 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:36.308986 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:36.309036 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:36.309064 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:36.309107 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:36.309140 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:36.340543 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.13 ********************************************************************************* I0920 05:49:36.344224 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:36.344329 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:36.344393 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:36.382396 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:36.539759 9338 executor.go:111] Tasks: 0 done / 100 total; 47 can run W0920 05:49:36.540133 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:36.542997 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:36.543783 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:36.545660 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0920 05:49:36.545798 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:36.545702 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:36.545867 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0920 05:49:36.548089 9338 executor.go:111] Tasks: 47 done / 100 total; 24 can run I0920 05:49:36.553891 9338 executor.go:111] Tasks: 71 done / 100 total; 23 can run I0920 05:49:36.554267 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:36.554446 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:36.554683 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:36.555805 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:36.557703 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:36.560872 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:36.561094 9338 executor.go:111] Tasks: 94 done / 100 total; 3 can run I0920 05:49:36.562333 9338 executor.go:111] Tasks: 97 done / 100 total; 3 can run I0920 05:49:36.562589 9338 executor.go:111] Tasks: 100 done / 100 total; 0 can run I0920 05:49:36.624446 9338 target.go:112] Terraform output is in /tmp/test2290841268/out I0920 05:49:36.629059 9338 featureflag.go:160] FeatureFlag "APIServerNodes"=false --- PASS: TestAPIServerNodes (0.57s) === RUN TestNTHQueueProcessor I0920 05:49:36.629184 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:36.629405 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:36.629460 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:36.629560 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:36.629616 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:36.629660 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:36.629690 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:36.629729 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:36.629760 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:36.654714 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0920 05:49:36.658077 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:36.658173 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:36.698584 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:36.829328 9338 executor.go:111] Tasks: 0 done / 101 total; 45 can run I0920 05:49:36.829900 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:36.829937 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:36.829964 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:36.830077 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:36.830343 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target W0920 05:49:36.830435 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:36.832921 9338 executor.go:111] Tasks: 45 done / 101 total; 24 can run I0920 05:49:36.842297 9338 executor.go:111] Tasks: 69 done / 101 total; 26 can run I0920 05:49:36.842657 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:36.842876 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:36.844090 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:36.845038 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:36.845161 9338 executor.go:111] Tasks: 95 done / 101 total; 2 can run I0920 05:49:36.845505 9338 executor.go:111] Tasks: 97 done / 101 total; 4 can run I0920 05:49:36.845794 9338 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0920 05:49:36.908473 9338 target.go:112] Terraform output is in /tmp/test447571761/out I0920 05:49:36.913272 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:36.913494 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:36.913550 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:36.913600 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:36.913653 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:36.913694 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:36.913725 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:36.913763 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:36.913790 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:36.938545 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0920 05:49:36.941914 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:36.942009 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:36.980886 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0920 05:49:37.104304 9338 executor.go:111] Tasks: 0 done / 101 total; 45 can run W0920 05:49:37.104712 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:37.104860 9338 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0920 05:49:37.104905 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:37.105008 9338 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0920 05:49:37.105108 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:37.105222 9338 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0920 05:49:37.105235 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:37.105362 9338 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0920 05:49:37.105323 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" }] }] } I0920 05:49:37.111917 9338 executor.go:111] Tasks: 45 done / 101 total; 24 can run I0920 05:49:37.117346 9338 executor.go:111] Tasks: 69 done / 101 total; 26 can run I0920 05:49:37.117849 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:37.118055 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:37.118917 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:37.120558 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:37.120697 9338 executor.go:111] Tasks: 95 done / 101 total; 2 can run I0920 05:49:37.120965 9338 executor.go:111] Tasks: 97 done / 101 total; 4 can run I0920 05:49:37.121230 9338 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0920 05:49:37.125098 9338 target.go:168] Cloudformation output is in /tmp/test2902286562/out --- PASS: TestNTHQueueProcessor (0.51s) === RUN TestCustomIRSA I0920 05:49:37.135908 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:37.136132 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:37.136185 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:37.136230 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:37.136280 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:37.136359 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:37.136391 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:37.136427 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:37.136453 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:37.169252 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0920 05:49:37.172886 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:37.172989 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:37.211615 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:37.754320 9338 executor.go:111] Tasks: 0 done / 100 total; 50 can run W0920 05:49:37.754575 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:37.754884 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:37.754916 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:37.754990 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:37.755301 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:37.755503 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:37.757397 9338 executor.go:111] Tasks: 50 done / 100 total; 25 can run I0920 05:49:37.763393 9338 executor.go:111] Tasks: 75 done / 100 total; 21 can run I0920 05:49:37.763789 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:37.764057 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:37.766760 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:37.770467 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:37.770610 9338 executor.go:111] Tasks: 96 done / 100 total; 2 can run I0920 05:49:37.770910 9338 executor.go:111] Tasks: 98 done / 100 total; 2 can run I0920 05:49:37.771132 9338 executor.go:111] Tasks: 100 done / 100 total; 0 can run I0920 05:49:37.825212 9338 target.go:112] Terraform output is in /tmp/test2039829809/out --- PASS: TestCustomIRSA (0.70s) === RUN TestClusterNameDigit I0920 05:49:37.833924 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:37.834157 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:37.834218 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:37.834270 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:37.834327 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:37.834419 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:37.834449 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:37.834485 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:37.834514 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:37.866327 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0920 05:49:37.870253 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:37.870356 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:37.909567 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:38.010655 9338 executor.go:111] Tasks: 0 done / 95 total; 47 can run I0920 05:49:38.010880 9338 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0920 05:49:38.010900 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:38.010961 9338 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0920 05:49:38.011130 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:38.011192 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.123.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0920 05:49:38.011593 9338 warmpool.go:124] ASG warm pool is not supported by the terraform target I0920 05:49:38.014062 9338 executor.go:111] Tasks: 47 done / 95 total; 23 can run I0920 05:49:38.017510 9338 executor.go:111] Tasks: 70 done / 95 total; 21 can run I0920 05:49:38.017781 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:38.017927 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:38.019023 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:38.019144 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:38.019220 9338 executor.go:111] Tasks: 91 done / 95 total; 2 can run I0920 05:49:38.019531 9338 executor.go:111] Tasks: 93 done / 95 total; 2 can run I0920 05:49:38.019714 9338 executor.go:111] Tasks: 95 done / 95 total; 0 can run I0920 05:49:38.075154 9338 target.go:112] Terraform output is in /tmp/test3956039242/out --- PASS: TestClusterNameDigit (0.25s) === RUN TestLifecycleMinimalAWS I0920 05:49:38.079661 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:38.079906 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:38.079995 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:38.080054 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:38.080115 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:38.080163 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:38.080194 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:38.080235 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:38.080265 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:38.080336 9338 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal.example.com I0920 05:49:38.104010 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:38.107267 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:38.107368 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0920 05:49:38.145641 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:38.244527 9338 iamrole.go:89] ListRoles: { } I0920 05:49:38.246159 9338 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0920 05:49:38.246728 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0920 05:49:38.246939 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0920 05:49:38.246932 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I0920 05:49:38.247203 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:38.247205 9338 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0920 05:49:38.247248 9338 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0920 05:49:38.247326 9338 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } I0920 05:49:38.247455 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0920 05:49:38.247523 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:38.247539 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } I0920 05:49:38.247597 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:38.247666 9338 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0920 05:49:38.247688 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0920 05:49:38.247928 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0920 05:49:38.248252 9338 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0920 05:49:38.248339 9338 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0920 05:49:38.248378 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0920 05:49:38.248439 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:38.248488 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0920 05:49:38.248668 9338 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0920 05:49:38.248824 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0920 05:49:38.248876 9338 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0920 05:49:38.248987 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0920 05:49:38.249737 9338 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0920 05:49:38.250078 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0920 05:49:38.250027 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0920 05:49:38.250254 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0920 05:49:38.250250 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0920 05:49:38.250397 9338 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0920 05:49:38.250495 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0920 05:49:38.250413 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0920 05:49:38.251405 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0920 05:49:38.251503 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:38.251596 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0920 05:49:38.251774 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:38.252076 9338 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0920 05:49:38.252398 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0920 05:49:38.252593 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0920 05:49:38.253256 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" } ] }], VpcId: "vpc-1" } I0920 05:49:38.253523 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:38.253862 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0920 05:49:38.254693 9338 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0920 05:49:38.254745 9338 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0920 05:49:38.254823 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0920 05:49:38.254888 9338 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0920 05:49:38.254965 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0920 05:49:38.255025 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:38.255073 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0920 05:49:38.255143 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:38.255996 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:38.255972 9338 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0920 05:49:38.256147 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0920 05:49:38.256234 9338 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] } I0920 05:49:38.257382 9338 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0920 05:49:38.259807 9338 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0920 05:49:38.260071 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:38.260266 9338 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0920 05:49:38.260349 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:38.260551 9338 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0920 05:49:38.260616 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0920 05:49:38.260875 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-2382to4000-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0920 05:49:38.261108 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0920 05:49:38.261250 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:38.261357 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-::/0" }] }] } I0920 05:49:38.261579 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0920 05:49:38.261740 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0920 05:49:38.261911 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0920 05:49:38.262064 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-4003to65535-masters.minimal.example.com" }] }] } I0920 05:49:38.262238 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:38.262367 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0920 05:49:38.262663 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-udp-1to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0920 05:49:38.262907 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0920 05:49:38.263122 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:38.263369 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:38.263586 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:38.263646 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0920 05:49:38.263832 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-1to2379-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0920 05:49:38.264041 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal.example.com" }] }] } I0920 05:49:38.264224 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0920 05:49:38.264289 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0920 05:49:38.266400 9338 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0920 05:49:38.266887 9338 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0920 05:49:38.267115 9338 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0920 05:49:38.267168 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:38.267210 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:38.267283 9338 dns.go:238] Pre-creating DNS records I0920 05:49:38.267315 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:38.273106 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:38.276319 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:38.276414 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0920 05:49:38.313992 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:38.418898 9338 iamrole.go:89] ListRoles: { } I0920 05:49:38.420499 9338 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0920 05:49:38.420796 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0920 05:49:38.421046 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0920 05:49:38.421225 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:38.421342 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:38.421557 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:38.421574 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0920 05:49:38.421752 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0920 05:49:38.421809 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0920 05:49:38.421894 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:38.421954 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0920 05:49:38.423411 9338 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0920 05:49:38.423535 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:38.423606 9338 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0920 05:49:38.423725 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0920 05:49:38.423678 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0920 05:49:38.423791 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0920 05:49:38.423956 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0920 05:49:38.424113 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:38.424170 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0920 05:49:38.424339 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0920 05:49:38.424519 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:38.424594 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:38.424695 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:38.428723 9338 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0920 05:49:38.428977 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:38.429161 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:38.429420 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:38.429637 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:38.429849 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:38.430140 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:38.430310 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:38.432313 9338 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0920 05:49:38.432584 9338 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0920 05:49:38.432739 9338 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0920 05:49:38.435544 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0920 05:49:38.435611 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0920 05:49:38.435648 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0920 05:49:38.435680 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0920 05:49:38.435716 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0920 05:49:38.435756 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0920 05:49:38.435790 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0920 05:49:38.435823 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0920 05:49:38.435859 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0920 05:49:38.435898 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0920 05:49:38.435937 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0920 05:49:38.435977 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0920 05:49:38.436019 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0920 05:49:38.436053 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0920 05:49:38.436086 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0920 05:49:38.436124 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0920 05:49:38.436154 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0920 05:49:38.436189 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0920 05:49:38.436223 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:38.436263 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0920 05:49:38.436297 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0920 05:49:38.436330 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0920 05:49:38.436363 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0920 05:49:38.436399 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0920 05:49:38.436458 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0920 05:49:38.436497 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0920 05:49:38.437856 9338 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0920 05:49:38.437884 9338 keypairs.go:95] DescribeKeyPairs: { } I0920 05:49:38.437924 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0920 05:49:38.437974 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0920 05:49:38.438035 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0920 05:49:38.438097 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0920 05:49:38.438144 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0920 05:49:38.438200 9338 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0920 05:49:38.438248 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0920 05:49:38.438287 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0920 05:49:38.438333 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0920 05:49:38.438387 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0920 05:49:38.438430 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } W0920 05:49:38.438475 9338 tags.go:48] unknown cluster tag on vpc:vpc-12345678: "kubernetes.io/cluster/minimal.example.com"="" W0920 05:49:38.438508 9338 api.go:53] PageSize not implemented I0920 05:49:38.438530 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0920 05:49:38.438563 9338 loadbalancers.go:34] PageSize not implemented I0920 05:49:38.438583 9338 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0920 05:49:38.438611 9338 targetgroups.go:35] PageSize not implemented I0920 05:49:38.438631 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0920 05:49:38.438686 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:38.438716 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:38.438767 9338 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0920 05:49:38.438824 9338 iamrole.go:89] ListRoles: { } I0920 05:49:38.438882 9338 internetgateways.go:101] DescribeInternetGateways: { } I0920 05:49:38.438930 9338 routetable.go:53] DescribeRouteTables: { } I0920 05:49:38.438964 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:38.439574 9338 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0920 05:49:38.439602 9338 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } I0920 05:49:38.439676 9338 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } autoscaling-group:nodes.minimal.example.com ok iam-instance-profile:nodes.minimal.example.com ok autoscaling-config:lt-2 ok autoscaling-config:lt-1 ok autoscaling-group:master-us-test-1a.masters.minimal.example.com ok volume:vol-1 ok I0920 05:49:38.439565 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.minimal.example.com. ok I0920 05:49:38.439822 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:38.439932 9338 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0920 05:49:38.440039 9338 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } iam-instance-profile:masters.minimal.example.com ok I0920 05:49:38.439917 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0920 05:49:38.440158 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:38.440189 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0920 05:49:38.440253 9338 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0920 05:49:38.440333 9338 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0920 05:49:38.440411 9338 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0920 05:49:38.440495 9338 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0920 05:49:38.440538 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:38.440587 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } W0920 05:49:38.440782 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:38.440797 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0920 05:49:38.440872 9338 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal.example.com" } I0920 05:49:38.440960 9338 iamrole.go:180] ListAttachedRolePolicies: masters.minimal.example.com I0920 05:49:38.440945 9338 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0920 05:49:38.440988 9338 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0920 05:49:38.441064 9338 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal.example.com" } iam-role:masters.minimal.example.com ok I0920 05:49:38.441109 9338 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal.example.com" } I0920 05:49:38.441154 9338 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal.example.com I0920 05:49:38.441169 9338 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0920 05:49:38.441196 9338 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal.example.com" } iam-role:nodes.minimal.example.com ok I0920 05:49:38.441264 9338 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0920 05:49:38.441349 9338 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0920 05:49:38.441424 9338 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0920 05:49:38.441694 9338 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleMinimalAWS (0.36s) === RUN TestLifecycleMinimalOpenstack I0920 05:49:38.441966 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster minimal-openstack.k8s.local I0920 05:49:38.452120 9338 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:38.522872 9338 apply_cluster.go:487] Gossip DNS: skipping DNS validation W0920 05:49:38.671571 9338 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0920 05:49:38.671627 9338 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0920 05:49:38.690175 9338 executor.go:111] Tasks: 0 done / 96 total; 42 can run I0920 05:49:38.695034 9338 executor.go:111] Tasks: 42 done / 96 total; 45 can run I0920 05:49:38.748959 9338 executor.go:111] Tasks: 87 done / 96 total; 5 can run I0920 05:49:38.752426 9338 executor.go:111] Tasks: 92 done / 96 total; 4 can run W0920 05:49:38.755737 9338 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0920 05:49:38.755825 9338 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0920 05:49:38.755957 9338 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I0920 05:49:38.761545 9338 executor.go:111] Tasks: 96 done / 96 total; 0 can run ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:38.809351 9338 apply_cluster.go:487] Gossip DNS: skipping DNS validation W0920 05:49:38.963378 9338 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0920 05:49:38.963430 9338 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0920 05:49:38.983186 9338 executor.go:111] Tasks: 0 done / 96 total; 42 can run I0920 05:49:38.990249 9338 executor.go:111] Tasks: 42 done / 96 total; 45 can run I0920 05:49:38.999215 9338 executor.go:111] Tasks: 87 done / 96 total; 5 can run I0920 05:49:39.001852 9338 executor.go:111] Tasks: 92 done / 96 total; 4 can run W0920 05:49:39.007017 9338 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0920 05:49:39.010883 9338 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0920 05:49:39.011156 9338 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I0920 05:49:39.012172 9338 executor.go:111] Tasks: 96 done / 96 total; 0 can run Instance:757be15f-04e0-4e82-9bf6-5d772f3014d4 ok Subnet:bf5e6f1e-2e6c-4ca9-8f34-e2f0250f37f9 ok Instance:ff35a766-6b68-426b-be96-7e9b9026536c ok Port:24f90ab8-b874-4645-8db8-f05dbcabd3a8 ok ServerGroup:64030e22-523e-400d-bc5d-bea454869fb0 ok ServerGroup:815c2a5b-28f1-48bb-8006-d0864af415e1 ok Network:7281aa9e-a174-4143-8b37-0e247defc7a4 ok Instance:42611f75-2cae-4a1c-849c-f394deed0527 ok SSHKey:kubernetes-minimal-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok SecurityGroup:d3cd6b01-e117-4e19-a108-5254aeb3ac74 ok SecurityGroup:55440123-0153-468a-b987-e66706c71918 ok SecurityGroup:6072b88f-7895-480e-bcfc-04fa7c686570 ok Port:87b7793b-0704-42b0-b906-7ac35962b143 ok Port:cfbeae86-715c-4f18-9cfd-60e0efdcd363 ok --- PASS: TestLifecycleMinimalOpenstack (1.68s) === RUN TestLifecycleMinimalGCE I0920 05:49:40.123833 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:486: running lifecycle test for cluster minimal-gce.example.com I0920 05:49:40.125720 9338 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W0920 05:49:40.353864 9338 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0920 05:49:40.353962 9338 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0920 05:49:40.354079 9338 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0920 05:49:40.354113 9338 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0920 05:49:40.354139 9338 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0920 05:49:40.354171 9338 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0920 05:49:40.354248 9338 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0920 05:49:40.355967 9338 executor.go:111] Tasks: 0 done / 64 total; 41 can run W0920 05:49:40.358956 9338 executor.go:139] error running task "ProjectIAMBinding/serviceaccount-nodes" (9s remaining to succeed): error updating IAM for project testproject: googleapi: Error 404: etag I0920 05:49:40.359048 9338 executor.go:111] Tasks: 40 done / 64 total; 18 can run I0920 05:49:40.364401 9338 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0920 05:49:40.367033 9338 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0920 05:49:40.367280 9338 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0920 05:49:40.367374 9338 dns.go:238] Pre-creating DNS records ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W0920 05:49:40.577755 9338 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0920 05:49:40.577856 9338 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0920 05:49:40.577995 9338 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0920 05:49:40.578040 9338 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0920 05:49:40.578068 9338 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0920 05:49:40.578112 9338 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0920 05:49:40.578214 9338 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0920 05:49:40.580001 9338 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0920 05:49:40.582590 9338 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0920 05:49:40.587778 9338 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0920 05:49:40.590507 9338 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0920 05:49:40.590696 9338 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0920 05:49:40.592379 9338 gce.go:98] Scanning zones: [us-test1-a] I0920 05:49:40.592779 9338 gce.go:1070] backend services not found, assuming none exist in project: "testproject" region: "us-test1" I0920 05:49:40.592807 9338 gce.go:1070] backend services not found, assuming none exist in project: "testproject" region: "us-test1" FirewallRule:node-to-master-minimal-gce-example-com ok Disk:d1-etcd-events-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/control-plane-minimal-g-fu1mg6@testproject.iam.gserviceaccount.com ok Disk:d1-etcd-main-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-ipv6-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-ipv6-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-master-us-test1-a-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-ipv6-minimal-gce-example-com ok FirewallRule:node-to-node-minimal-gce-example-com ok FirewallRule:master-to-master-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-nodes-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-ipv6-minimal-gce-example-com ok FirewallRule:master-to-node-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/node-minimal-gce-example-com@testproject.iam.gserviceaccount.com ok InstanceTemplate:nodes-minimal-gce-example-com-1663652980 ok InstanceTemplate:master-us-test1-a-minimal-do16cp-1663652980 ok Subnet:us-test1-minimal-gce-example-com ok Network:minimal-gce-example-com ok --- PASS: TestLifecycleMinimalGCE (0.47s) === RUN TestLifecycleFloatingIPOpenstack I0920 05:49:40.594492 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster floatingip-openstack.k8s.local I0920 05:49:40.602971 9338 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:40.668849 9338 apply_cluster.go:487] Gossip DNS: skipping DNS validation W0920 05:49:40.815498 9338 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0920 05:49:40.815559 9338 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0920 05:49:40.832324 9338 executor.go:111] Tasks: 0 done / 98 total; 43 can run I0920 05:49:40.842345 9338 executor.go:111] Tasks: 43 done / 98 total; 45 can run I0920 05:49:40.903035 9338 executor.go:111] Tasks: 88 done / 98 total; 6 can run I0920 05:49:40.906036 9338 executor.go:111] Tasks: 94 done / 98 total; 4 can run W0920 05:49:40.908409 9338 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0920 05:49:40.908745 9338 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0920 05:49:40.908878 9338 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I0920 05:49:40.913635 9338 executor.go:111] Tasks: 98 done / 98 total; 0 can run ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:40.960523 9338 apply_cluster.go:487] Gossip DNS: skipping DNS validation W0920 05:49:41.106760 9338 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0920 05:49:41.106806 9338 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0920 05:49:41.126383 9338 executor.go:111] Tasks: 0 done / 98 total; 43 can run I0920 05:49:41.135341 9338 executor.go:111] Tasks: 43 done / 98 total; 45 can run I0920 05:49:41.142457 9338 executor.go:111] Tasks: 88 done / 98 total; 6 can run I0920 05:49:41.149308 9338 executor.go:111] Tasks: 94 done / 98 total; 4 can run W0920 05:49:41.153177 9338 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0920 05:49:41.153188 9338 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0920 05:49:41.153437 9338 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I0920 05:49:41.155318 9338 executor.go:111] Tasks: 98 done / 98 total; 0 can run ServerGroup:aad881b7-15d6-43d6-9d41-aa099cbfc9f4 ok Router-IF:44d4b92b-5919-4f56-a0c4-a285d9165000 ok SSHKey:kubernetes-floatingip-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok Port:a7c0e701-6126-4879-8ca0-2e2fdd6196e5 ok SecurityGroup:f6668604-d2b7-4da5-acae-1fd844c34ed6 ok Port:85c5b67a-dd31-415f-8968-9260467477d3 ok Instance:20a2cbc4-ea0a-4d4d-a856-0ab2c192ba2f ok Port:87163109-1df2-4909-8a16-3cba68dc985e ok Router:4ec0f368-aacd-4f99-ab16-8a9fe8b9ca3a ok Network:c8c1aa71-7f01-47bc-8d93-2ddb245d3ec7 ok Subnet:44d4b92b-5919-4f56-a0c4-a285d9165000 ok ServerGroup:399f6d73-6cf9-4472-ba6d-525b05a73963 ok SecurityGroup:a4347439-f5a8-4c47-866f-d7094de6bbdf ok Instance:735d483e-046a-43e6-a2a6-c71fdafda834 ok SecurityGroup:1cd3c391-0584-4740-a6aa-c73f3ddadbe5 ok Port:b6bd34f4-4ccc-4240-9bfe-fc49e4b81ddb ok Instance:1e5507d1-8b17-4d57-8d99-5b1d780ae4ea ok --- PASS: TestLifecycleFloatingIPOpenstack (1.66s) === RUN TestLifecyclePrivateCalico I0920 05:49:42.257400 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:42.257687 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:42.257743 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:42.257789 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:42.257839 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:42.257884 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:42.257912 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:42.257950 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:42.257977 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:42.258053 9338 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatecalico.example.com I0920 05:49:42.287320 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0920 05:49:42.291410 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:42.291515 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:42.291580 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:42.330574 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:42.790424 9338 iamrole.go:89] ListRoles: { } I0920 05:49:42.791992 9338 executor.go:111] Tasks: 0 done / 127 total; 50 can run I0920 05:49:42.792139 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:42.792273 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } I0920 05:49:42.792440 9338 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0920 05:49:42.794157 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } I0920 05:49:42.794401 9338 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0920 05:49:42.794874 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } I0920 05:49:42.794982 9338 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", Tags: [{ Key: "Name", Value: "bastions.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0920 05:49:42.792551 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:42.798108 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:42.798238 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0920 05:49:42.798387 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0920 05:49:42.798513 9338 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "privatecalico.example.com" }] }] } I0920 05:49:42.798672 9338 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0920 05:49:42.798723 9338 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0920 05:49:42.798772 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0920 05:49:42.798914 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0920 05:49:42.799074 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0920 05:49:42.799205 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0920 05:49:42.799281 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:42.799337 9338 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0920 05:49:42.799565 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0920 05:49:42.799630 9338 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0920 05:49:42.799762 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0920 05:49:42.799814 9338 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-events.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0920 05:49:42.799951 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0920 05:49:42.800061 9338 executor.go:111] Tasks: 50 done / 127 total; 28 can run I0920 05:49:42.800212 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["privatecalico.example.com"] }] } I0920 05:49:42.800368 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0920 05:49:42.800388 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0920 05:49:42.800544 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I0920 05:49:42.800599 9338 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I0920 05:49:42.800557 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0920 05:49:42.800727 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0920 05:49:42.800809 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0920 05:49:42.800980 9338 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I0920 05:49:42.800968 9338 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0920 05:49:42.800883 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "Name", Value: "utility-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0920 05:49:42.801067 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0920 05:49:42.800985 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0920 05:49:42.801193 9338 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0920 05:49:42.801293 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0920 05:49:42.801628 9338 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0920 05:49:42.801802 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0920 05:49:42.801785 9338 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0920 05:49:42.802000 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I0920 05:49:42.802138 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.privatecalico.example.com" } ] }], VpcId: "vpc-1" } I0920 05:49:42.802159 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0920 05:49:42.802341 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0920 05:49:42.802518 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:42.802647 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:42.802785 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0920 05:49:42.802934 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0920 05:49:42.803114 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0920 05:49:42.802819 9338 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0920 05:49:42.803240 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastion-elb.privatecalico.example.com" }] }], VpcId: "vpc-1" } I0920 05:49:42.803370 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0920 05:49:42.803436 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0920 05:49:42.803487 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0920 05:49:42.803539 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatecalico.example.com" }] }], VpcId: "vpc-1" } I0920 05:49:42.803625 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0920 05:49:42.803670 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0920 05:49:42.803640 9338 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I0920 05:49:42.803719 9338 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0920 05:49:42.803804 9338 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0920 05:49:42.803848 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0920 05:49:42.803958 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0920 05:49:42.804020 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0920 05:49:42.804067 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0920 05:49:42.804120 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:42.804161 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0920 05:49:42.804217 9338 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" } ] } I0920 05:49:42.804300 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:42.804345 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0920 05:49:42.804401 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0920 05:49:42.804466 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0920 05:49:42.804551 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0920 05:49:42.804624 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0920 05:49:42.804700 9338 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" } ] } I0920 05:49:42.804832 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:42.804915 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:42.806120 9338 executor.go:111] Tasks: 78 done / 127 total; 40 can run W0920 05:49:42.806304 9338 api.go:53] PageSize not implemented I0920 05:49:42.806305 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0920 05:49:42.806418 9338 api.go:53] PageSize not implemented I0920 05:49:42.806454 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:42.806474 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0920 05:49:42.806592 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I0920 05:49:42.806703 9338 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"], Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] } I0920 05:49:42.806853 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0920 05:49:42.806893 9338 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0920 05:49:42.806924 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:42.806953 9338 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0920 05:49:42.807051 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:42.807059 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0920 05:49:42.807132 9338 tags.go:74] AddTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"], Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "api.privatecalico.example.com" }] } I0920 05:49:42.807198 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0920 05:49:42.807231 9338 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0920 05:49:42.807161 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-2382to4000-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0920 05:49:42.807282 9338 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0920 05:49:42.807490 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-1to2379-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0920 05:49:42.808038 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0920 05:49:42.808236 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:42.808349 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:42.808522 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0920 05:49:42.808721 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:42.808804 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0920 05:49:42.808843 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:42.808914 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0920 05:49:42.809175 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0920 05:49:42.809327 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0920 05:49:42.809404 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0920 05:49:42.809659 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0920 05:49:42.809890 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0920 05:49:42.810084 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0920 05:49:42.810236 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-udp-1to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0920 05:49:42.810407 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } I0920 05:49:42.810531 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:42.810635 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:42.810805 9338 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0920 05:49:42.810780 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I0920 05:49:42.810985 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-::/0" }] }] } I0920 05:49:42.811132 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0920 05:49:42.811168 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0920 05:49:42.811350 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-4-0to0-masters.privatecalico.example.com" }] }] } I0920 05:49:42.811515 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0920 05:49:42.811632 9338 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0920 05:49:42.811699 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-::/0" }] }] } I0920 05:49:42.811857 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-ingress-tcp-22to22-bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0920 05:49:42.812034 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0920 05:49:42.812189 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0920 05:49:42.812342 9338 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0920 05:49:42.812384 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0920 05:49:42.812432 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0920 05:49:42.812599 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0920 05:49:42.812790 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0920 05:49:42.812956 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0920 05:49:42.813083 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0920 05:49:42.813258 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-4003to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0920 05:49:42.813444 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:42.813635 9338 executor.go:111] Tasks: 118 done / 127 total; 5 can run I0920 05:49:42.813779 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0920 05:49:42.813821 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:42.813974 9338 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:42.813972 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0920 05:49:42.814163 9338 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0920 05:49:42.814277 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0920 05:49:42.814330 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:42.814379 9338 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0920 05:49:42.814457 9338 executor.go:111] Tasks: 123 done / 127 total; 4 can run I0920 05:49:42.814583 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:42.814732 9338 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0920 05:49:42.814808 9338 executor.go:111] Tasks: 127 done / 127 total; 0 can run I0920 05:49:42.814872 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:42.814907 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:42.814959 9338 dns.go:238] Pre-creating DNS records I0920 05:49:42.814985 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:42.821978 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0920 05:49:42.827591 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:42.827843 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:42.827983 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:42.866121 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:43.319076 9338 iamrole.go:89] ListRoles: { } I0920 05:49:43.321899 9338 executor.go:111] Tasks: 0 done / 127 total; 50 can run I0920 05:49:43.322491 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:43.322572 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0920 05:49:43.322747 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:43.322769 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0920 05:49:43.322912 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:43.323042 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0920 05:49:43.323171 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0920 05:49:43.323264 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0920 05:49:43.323288 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0920 05:49:43.323350 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:43.324846 9338 executor.go:111] Tasks: 50 done / 127 total; 28 can run I0920 05:49:43.324965 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I0920 05:49:43.325038 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0920 05:49:43.325155 9338 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I0920 05:49:43.325218 9338 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I0920 05:49:43.325232 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:43.325347 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0920 05:49:43.325597 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:43.325705 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0920 05:49:43.325943 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0920 05:49:43.326019 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["privatecalico.example.com"] }] } I0920 05:49:43.326208 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0920 05:49:43.326375 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:43.326425 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0920 05:49:43.326530 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0920 05:49:43.326630 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I0920 05:49:43.326759 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0920 05:49:43.326870 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0920 05:49:43.326982 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:43.327058 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:43.327100 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0920 05:49:43.327176 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0920 05:49:43.327232 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0920 05:49:43.330621 9338 executor.go:111] Tasks: 78 done / 127 total; 40 can run I0920 05:49:43.330855 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } W0920 05:49:43.331005 9338 api.go:53] PageSize not implemented I0920 05:49:43.331043 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I0920 05:49:43.331075 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:43.331168 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0920 05:49:43.331209 9338 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0920 05:49:43.331203 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0920 05:49:43.331318 9338 api.go:53] PageSize not implemented I0920 05:49:43.331291 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:43.331343 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I0920 05:49:43.331424 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0920 05:49:43.331460 9338 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatecalico-example-0uch4k" } I0920 05:49:43.331444 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:43.331601 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:43.331779 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:43.332102 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:43.332354 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:43.332481 9338 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0920 05:49:43.332530 9338 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0920 05:49:43.332582 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:43.332682 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:43.332812 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0920 05:49:43.335115 9338 executor.go:111] Tasks: 118 done / 127 total; 5 can run W0920 05:49:43.335227 9338 api.go:53] PageSize not implemented I0920 05:49:43.335254 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I0920 05:49:43.335284 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:43.335364 9338 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0920 05:49:43.335320 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0920 05:49:43.335425 9338 dnsname.go:121] AliasTarget for "api.privatecalico.example.com" is "api-privatecalico-example-0uch4k.elb.cloudmock.com" W0920 05:49:43.335392 9338 api.go:53] PageSize not implemented I0920 05:49:43.335470 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I0920 05:49:43.335588 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0920 05:49:43.335671 9338 executor.go:111] Tasks: 123 done / 127 total; 4 can run I0920 05:49:43.335791 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:43.335903 9338 executor.go:111] Tasks: 127 done / 127 total; 0 can run I0920 05:49:43.339577 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0920 05:49:43.339644 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0920 05:49:43.339684 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0920 05:49:43.339721 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0920 05:49:43.339758 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0920 05:49:43.339799 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0920 05:49:43.339837 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0920 05:49:43.339871 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0920 05:49:43.339905 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0920 05:49:43.339940 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0920 05:49:43.339997 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0920 05:49:43.340041 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0920 05:49:43.340083 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0920 05:49:43.340122 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0920 05:49:43.340162 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0920 05:49:43.340199 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0920 05:49:43.340232 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0920 05:49:43.340268 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0920 05:49:43.340309 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0920 05:49:43.340344 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0920 05:49:43.340381 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0920 05:49:43.340418 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0920 05:49:43.340455 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0920 05:49:43.340492 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0920 05:49:43.340529 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0920 05:49:43.340567 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:43.340608 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0920 05:49:43.340643 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0920 05:49:43.340682 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0920 05:49:43.340717 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0920 05:49:43.340752 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0920 05:49:43.340791 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0920 05:49:43.340829 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0920 05:49:43.340867 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0920 05:49:43.342498 9338 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0920 05:49:43.342530 9338 keypairs.go:95] DescribeKeyPairs: { } I0920 05:49:43.342571 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0920 05:49:43.342639 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0920 05:49:43.342715 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0920 05:49:43.342776 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0920 05:49:43.342829 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0920 05:49:43.342884 9338 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0920 05:49:43.342926 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0920 05:49:43.342972 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0920 05:49:43.343034 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0920 05:49:43.343091 9338 routetable.go:53] DescribeRouteTables: { } I0920 05:49:43.343119 9338 natgateway.go:117] DescribeNatGateways: { } I0920 05:49:43.343155 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0920 05:49:43.343198 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } W0920 05:49:43.343254 9338 api.go:53] PageSize not implemented I0920 05:49:43.343272 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I0920 05:49:43.343343 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0920 05:49:43.343374 9338 loadbalancers.go:34] PageSize not implemented I0920 05:49:43.343393 9338 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0920 05:49:43.343418 9338 targetgroups.go:35] PageSize not implemented I0920 05:49:43.343436 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0920 05:49:43.343489 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:43.343516 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:43.343559 9338 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0920 05:49:43.343606 9338 iamrole.go:89] ListRoles: { } I0920 05:49:43.343655 9338 internetgateways.go:101] DescribeInternetGateways: { } I0920 05:49:43.343702 9338 routetable.go:53] DescribeRouteTables: { } I0920 05:49:43.343743 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0920 05:49:43.343781 9338 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0920 05:49:43.343815 9338 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0920 05:49:43.344467 9338 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0920 05:49:43.344551 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-group:master-us-test-1a.masters.privatecalico.example.com ok autoscaling-group:nodes.privatecalico.example.com ok I0920 05:49:43.344568 9338 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0920 05:49:43.344680 9338 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } iam-instance-profile:masters.privatecalico.example.com ok I0920 05:49:43.344612 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } autoscaling-group:bastion.privatecalico.example.com ok W0920 05:49:43.344738 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:43.344506 9338 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatecalico-example-0uch4k" } I0920 05:49:43.344756 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } load-balancer:api-privatecalico-example-0uch4k ok security-group:sg-2 ok I0920 05:49:43.344782 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0920 05:49:43.344776 9338 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0920 05:49:43.344804 9338 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } load-balancer:bastion-privatecalico-exa-hocohm ok I0920 05:49:43.344864 9338 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } iam-instance-profile:bastions.privatecalico.example.com ok I0920 05:49:43.344832 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } W0920 05:49:43.344916 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:43.344932 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0920 05:49:43.344757 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:43.344927 9338 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.privatecalico.example.com. ok I0920 05:49:43.345000 9338 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } iam-instance-profile:nodes.privatecalico.example.com ok I0920 05:49:43.344974 9338 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0920 05:49:43.345059 9338 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0920 05:49:43.345099 9338 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-2 ok autoscaling-config:lt-3 ok I0920 05:49:43.345184 9338 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-1 ok I0920 05:49:43.345251 9338 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0920 05:49:43.345309 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0920 05:49:43.345361 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [ { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0920 05:49:43.345610 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:43.345625 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0920 05:49:43.345722 9338 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatecalico.example.com" } I0920 05:49:43.345776 9338 iamrole.go:180] ListAttachedRolePolicies: bastions.privatecalico.example.com I0920 05:49:43.345742 9338 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } I0920 05:49:43.345791 9338 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } subnet:subnet-2 ok I0920 05:49:43.345843 9338 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatecalico.example.com" } iam-role:bastions.privatecalico.example.com ok I0920 05:49:43.345840 9338 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I0920 05:49:43.345878 9338 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatecalico.example.com" } I0920 05:49:43.345941 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0920 05:49:43.345981 9338 iamrole.go:180] ListAttachedRolePolicies: nodes.privatecalico.example.com I0920 05:49:43.346003 9338 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0920 05:49:43.346002 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }] } I0920 05:49:43.346061 9338 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatecalico.example.com" } W0920 05:49:43.346119 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:43.346138 9338 iamrole.go:180] ListAttachedRolePolicies: masters.privatecalico.example.com I0920 05:49:43.346134 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } I0920 05:49:43.346167 9338 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } security-group:sg-3 ok I0920 05:49:43.346214 9338 iamrole.go:137] DeleteRole: { RoleName: "masters.privatecalico.example.com" } iam-role:masters.privatecalico.example.com ok I0920 05:49:43.346257 9338 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatecalico.example.com" } iam-role:nodes.privatecalico.example.com ok I0920 05:49:43.346227 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:43.346314 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 } ] } W0920 05:49:43.346506 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:43.346527 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0920 05:49:43.346562 9338 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0920 05:49:43.346670 9338 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0920 05:49:43.346733 9338 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0920 05:49:43.346810 9338 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0920 05:49:43.346884 9338 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0920 05:49:43.347169 9338 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateCalico (1.09s) === RUN TestLifecyclePrivateKopeio I0920 05:49:43.347471 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:43.347706 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:43.347750 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:43.347791 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:43.347839 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:43.347886 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:43.347916 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:43.347952 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:43.347976 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:43.348037 9338 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatekopeio.example.com I0920 05:49:43.378511 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:43.384668 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:43.384769 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:43.384838 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:43.422967 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:43.532012 9338 iamrole.go:89] ListRoles: { } I0920 05:49:43.535165 9338 executor.go:111] Tasks: 0 done / 126 total; 47 can run I0920 05:49:43.535697 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.535864 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:43.536054 9338 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0920 05:49:43.536332 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0920 05:49:43.536082 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } I0920 05:49:43.536474 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.536632 9338 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0920 05:49:43.536928 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:43.537114 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } I0920 05:49:43.537073 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:43.536831 9338 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "privatekopeio.example.com" }] }] } I0920 05:49:43.537313 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.538067 9338 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0920 05:49:43.537258 9338 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "bastions.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0920 05:49:43.538241 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0920 05:49:43.538306 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.538445 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } I0920 05:49:43.538485 9338 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0920 05:49:43.538601 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0920 05:49:43.538583 9338 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0920 05:49:43.538664 9338 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0920 05:49:43.538698 9338 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0920 05:49:43.538727 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0920 05:49:43.538765 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:43.539288 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0920 05:49:43.540937 9338 executor.go:111] Tasks: 47 done / 126 total; 31 can run I0920 05:49:43.541070 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:43.541208 9338 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0920 05:49:43.541215 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.541346 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1b"] }] } I0920 05:49:43.541480 9338 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I0920 05:49:43.541403 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.541539 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0920 05:49:43.541474 9338 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" }, { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0920 05:49:43.541582 9338 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0920 05:49:43.541600 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0920 05:49:43.541642 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:43.541732 9338 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0920 05:49:43.541667 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0920 05:49:43.541838 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:43.541891 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:43.541882 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0920 05:49:43.541938 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0920 05:49:43.542136 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.542143 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0920 05:49:43.542319 9338 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0920 05:49:43.542380 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0920 05:49:43.542441 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.542664 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0920 05:49:43.542695 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "Name", Value: "utility-us-test-1a.privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0920 05:49:43.542829 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0920 05:49:43.542881 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0920 05:49:43.542940 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0920 05:49:43.543049 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0920 05:49:43.543096 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:43.543135 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.543341 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.8.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0920 05:49:43.543345 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.543466 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0920 05:49:43.543586 9338 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0920 05:49:43.543547 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.543704 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0920 05:49:43.543790 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0920 05:49:43.543841 9338 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0920 05:49:43.543938 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0920 05:49:43.543985 9338 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [ { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] } I0920 05:49:43.544068 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0920 05:49:43.544179 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I0920 05:49:43.544300 9338 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0920 05:49:43.544335 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0920 05:49:43.544440 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.544554 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0920 05:49:43.544181 9338 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0920 05:49:43.544626 9338 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] } I0920 05:49:43.544709 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0920 05:49:43.544762 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0920 05:49:43.544801 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0920 05:49:43.544852 9338 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" } ] } I0920 05:49:43.544928 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0920 05:49:43.545085 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0920 05:49:43.545163 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0920 05:49:43.545246 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0920 05:49:43.545467 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0920 05:49:43.545564 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0920 05:49:43.545642 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0920 05:49:43.545688 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0920 05:49:43.545729 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0920 05:49:43.545818 9338 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0920 05:49:43.545914 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:43.545968 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0920 05:49:43.546016 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0920 05:49:43.546071 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0920 05:49:43.546112 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0920 05:49:43.546787 9338 executor.go:111] Tasks: 78 done / 126 total; 39 can run W0920 05:49:43.546935 9338 api.go:53] PageSize not implemented I0920 05:49:43.547014 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0920 05:49:43.547097 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0920 05:49:43.547198 9338 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"], Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0920 05:49:43.547267 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0920 05:49:43.547215 9338 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0920 05:49:43.547302 9338 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0920 05:49:43.547360 9338 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } W0920 05:49:43.547439 9338 api.go:53] PageSize not implemented I0920 05:49:43.547459 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0920 05:49:43.547457 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0920 05:49:43.547546 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0920 05:49:43.547689 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0920 05:49:43.547747 9338 tags.go:74] AddTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"], Tags: [{ Key: "Name", Value: "api.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0920 05:49:43.547825 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0920 05:49:43.547834 9338 tags.go:42] CreateTags { Resources: ["nat-a2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0920 05:49:43.547860 9338 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0920 05:49:43.547912 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0920 05:49:43.547960 9338 tags.go:42] CreateTags { Resources: ["subnet-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0920 05:49:43.547935 9338 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0920 05:49:43.548004 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0920 05:49:43.548044 9338 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0920 05:49:43.548084 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatekopeio.example.com" }] }] } I0920 05:49:43.548335 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:43.548456 9338 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0920 05:49:43.548663 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:43.548849 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0920 05:49:43.549154 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0920 05:49:43.549233 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0920 05:49:43.549399 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:43.549497 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:43.549629 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0920 05:49:43.549838 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:43.549965 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-1to2379-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0920 05:49:43.550165 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-udp-1to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0920 05:49:43.550310 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0920 05:49:43.550459 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0920 05:49:43.550594 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0920 05:49:43.550742 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:43.550840 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } I0920 05:49:43.550943 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0920 05:49:43.551075 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:43.551158 9338 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0920 05:49:43.551210 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:43.551309 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0920 05:49:43.551442 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" }] }] } I0920 05:49:43.551577 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0920 05:49:43.551617 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0920 05:49:43.551716 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I0920 05:49:43.551823 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-ingress-tcp-22to22-bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0920 05:49:43.551954 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-2382to4000-masters.privatekopeio.example.com" }] }] } I0920 05:49:43.552084 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-4"] }] } I0920 05:49:43.552125 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0920 05:49:43.552256 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0920 05:49:43.552299 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0920 05:49:43.552339 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0920 05:49:43.552465 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0920 05:49:43.552506 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-4003to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0920 05:49:43.552640 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0920 05:49:43.552767 9338 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0920 05:49:43.552809 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0920 05:49:43.552936 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:43.552983 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:43.553048 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:43.553125 9338 tags.go:42] CreateTags { Resources: ["nat-b2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0920 05:49:43.553213 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-4" } I0920 05:49:43.553284 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-2" } I0920 05:49:43.553383 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0920 05:49:43.553472 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-3" } I0920 05:49:43.553610 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0920 05:49:43.553701 9338 tags.go:42] CreateTags { Resources: ["subnet-4"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0920 05:49:43.553772 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0920 05:49:43.553845 9338 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0920 05:49:43.555697 9338 executor.go:111] Tasks: 117 done / 126 total; 6 can run I0920 05:49:43.555825 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0920 05:49:43.555849 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:43.555992 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0920 05:49:43.556069 9338 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-b2345678", RouteTableId: "rtb-1" } I0920 05:49:43.556153 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0920 05:49:43.556283 9338 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-a2345678", RouteTableId: "rtb-3" } I0920 05:49:43.556353 9338 executor.go:111] Tasks: 123 done / 126 total; 3 can run I0920 05:49:43.556526 9338 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0920 05:49:43.556581 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:43.556618 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:43.556710 9338 dns.go:238] Pre-creating DNS records I0920 05:49:43.556739 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:43.563447 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:43.567120 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:43.567218 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:43.567287 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:43.605228 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:43.720740 9338 iamrole.go:89] ListRoles: { } I0920 05:49:43.723435 9338 executor.go:111] Tasks: 0 done / 126 total; 47 can run I0920 05:49:43.723771 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.723928 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:43.724017 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0920 05:49:43.724083 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0920 05:49:43.724163 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.724327 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:43.724373 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] }] } I0920 05:49:43.724495 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.724607 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:43.724665 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:43.726748 9338 executor.go:111] Tasks: 47 done / 126 total; 31 can run I0920 05:49:43.726943 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.727155 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0920 05:49:43.727150 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I0920 05:49:43.727249 9338 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I0920 05:49:43.727300 9338 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0920 05:49:43.727298 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.727440 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0920 05:49:43.727482 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0920 05:49:43.727628 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0920 05:49:43.727695 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:43.727868 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.728030 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:43.728092 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0920 05:49:43.728373 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:43.728482 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.728662 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.728801 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0920 05:49:43.728842 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.728959 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0920 05:49:43.729099 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:43.729157 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] }] } I0920 05:49:43.729363 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0920 05:49:43.729410 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.729562 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:43.729661 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0920 05:49:43.729817 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0920 05:49:43.729885 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0920 05:49:43.731805 9338 executor.go:111] Tasks: 78 done / 126 total; 39 can run I0920 05:49:43.731945 9338 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } W0920 05:49:43.732112 9338 api.go:53] PageSize not implemented I0920 05:49:43.732090 9338 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0920 05:49:43.732153 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0920 05:49:43.732321 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0920 05:49:43.732319 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:43.732388 9338 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } W0920 05:49:43.732557 9338 api.go:53] PageSize not implemented I0920 05:49:43.732564 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:43.732585 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0920 05:49:43.732692 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0920 05:49:43.732733 9338 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0920 05:49:43.732725 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:43.732935 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:43.733084 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:43.733195 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:43.733338 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:43.733478 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:43.733602 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0920 05:49:43.733719 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:43.733854 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:43.734065 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:43.736234 9338 executor.go:111] Tasks: 117 done / 126 total; 6 can run W0920 05:49:43.736334 9338 api.go:53] PageSize not implemented I0920 05:49:43.736357 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0920 05:49:43.736412 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } W0920 05:49:43.736496 9338 api.go:53] PageSize not implemented I0920 05:49:43.736514 9338 dnsname.go:121] AliasTarget for "api.privatekopeio.example.com" is "api-privatekopeio-example-tl2bv8.elb.cloudmock.com" I0920 05:49:43.736470 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:43.736534 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0920 05:49:43.736598 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0920 05:49:43.736769 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0920 05:49:43.736883 9338 executor.go:111] Tasks: 123 done / 126 total; 3 can run I0920 05:49:43.737049 9338 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0920 05:49:43.740727 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0920 05:49:43.740810 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0920 05:49:43.740882 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0920 05:49:43.740947 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0920 05:49:43.741017 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0920 05:49:43.741090 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0920 05:49:43.741157 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0920 05:49:43.741221 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0920 05:49:43.741285 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0920 05:49:43.741357 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0920 05:49:43.741418 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0920 05:49:43.741487 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0920 05:49:43.741554 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0920 05:49:43.741630 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0920 05:49:43.741684 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0920 05:49:43.741747 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0920 05:49:43.741811 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0920 05:49:43.741869 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0920 05:49:43.741937 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0920 05:49:43.742007 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0920 05:49:43.742071 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0920 05:49:43.742141 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0920 05:49:43.742204 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0920 05:49:43.742269 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0920 05:49:43.742332 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:43.742401 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0920 05:49:43.742459 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0920 05:49:43.742533 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0920 05:49:43.742597 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0920 05:49:43.742664 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0920 05:49:43.742719 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0920 05:49:43.742784 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0920 05:49:43.742848 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0920 05:49:43.742915 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0920 05:49:43.742976 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0920 05:49:43.744762 9338 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0920 05:49:43.744799 9338 keypairs.go:95] DescribeKeyPairs: { } I0920 05:49:43.744857 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.744948 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0920 05:49:43.745059 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.745148 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.745220 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.745302 9338 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.745358 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.745440 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0920 05:49:43.745542 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.745639 9338 routetable.go:53] DescribeRouteTables: { } I0920 05:49:43.745695 9338 natgateway.go:117] DescribeNatGateways: { } I0920 05:49:43.745757 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0920 05:49:43.745816 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } W0920 05:49:43.745902 9338 api.go:53] PageSize not implemented I0920 05:49:43.745930 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0920 05:49:43.746017 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0920 05:49:43.746056 9338 loadbalancers.go:34] PageSize not implemented I0920 05:49:43.746084 9338 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0920 05:49:43.746123 9338 targetgroups.go:35] PageSize not implemented I0920 05:49:43.746150 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0920 05:49:43.746227 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:43.746272 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:43.746352 9338 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0920 05:49:43.746412 9338 iamrole.go:89] ListRoles: { } I0920 05:49:43.746476 9338 internetgateways.go:101] DescribeInternetGateways: { } I0920 05:49:43.746545 9338 routetable.go:53] DescribeRouteTables: { } I0920 05:49:43.746621 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3","rtb-2","rtb-1"] } I0920 05:49:43.746711 9338 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0920 05:49:43.746768 9338 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0920 05:49:43.747470 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } autoscaling-group:master-us-test-1a.masters.privatekopeio.example.com ok autoscaling-config:lt-1 ok autoscaling-group:bastion.privatekopeio.example.com ok I0920 05:49:43.747569 9338 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } load-balancer:bastion-privatekopeio-exa-d8ef8e ok I0920 05:49:43.747660 9338 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } load-balancer:api-privatekopeio-example-tl2bv8 ok autoscaling-group:nodes.privatekopeio.example.com ok I0920 05:49:43.747653 9338 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0920 05:49:43.747745 9338 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } iam-instance-profile:nodes.privatekopeio.example.com ok I0920 05:49:43.747795 9338 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0920 05:49:43.747634 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.privatekopeio.example.com. ok I0920 05:49:43.747850 9338 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } I0920 05:49:43.747555 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } iam-instance-profile:masters.privatekopeio.example.com ok W0920 05:49:43.747896 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:43.747873 9338 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0920 05:49:43.747936 9338 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } iam-instance-profile:bastions.privatekopeio.example.com ok I0920 05:49:43.747933 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok autoscaling-config:lt-3 ok I0920 05:49:43.748023 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0920 05:49:43.748112 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W0920 05:49:43.748250 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:43.748272 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0920 05:49:43.748329 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:43.748413 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } W0920 05:49:43.748512 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:43.748534 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0920 05:49:43.748604 9338 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-2 ok I0920 05:49:43.748706 9338 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0920 05:49:43.748803 9338 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0920 05:49:43.748856 9338 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0920 05:49:43.748932 9338 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0920 05:49:43.749083 9338 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I0920 05:49:43.749112 9338 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatekopeio.example.com" } I0920 05:49:43.749152 9338 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-4" } subnet:subnet-4 ok I0920 05:49:43.749199 9338 iamrole.go:180] ListAttachedRolePolicies: masters.privatekopeio.example.com I0920 05:49:43.749234 9338 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0920 05:49:43.749255 9338 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0920 05:49:43.749309 9338 iamrole.go:137] DeleteRole: { RoleName: "masters.privatekopeio.example.com" } iam-role:masters.privatekopeio.example.com ok I0920 05:49:43.749311 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:43.749358 9338 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatekopeio.example.com" } I0920 05:49:43.749413 9338 iamrole.go:180] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0920 05:49:43.749427 9338 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0920 05:49:43.749464 9338 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatekopeio.example.com" } iam-role:bastions.privatekopeio.example.com ok I0920 05:49:43.749497 9338 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatekopeio.example.com" } I0920 05:49:43.749397 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } W0920 05:49:43.749568 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:43.749594 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } I0920 05:49:43.749544 9338 iamrole.go:180] ListAttachedRolePolicies: nodes.privatekopeio.example.com security-group:sg-1 ok I0920 05:49:43.749638 9338 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0920 05:49:43.749672 9338 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatekopeio.example.com" } iam-role:nodes.privatekopeio.example.com ok I0920 05:49:43.749658 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0920 05:49:43.749748 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0920 05:49:43.749847 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:43.749868 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0920 05:49:43.749922 9338 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0920 05:49:43.750060 9338 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0920 05:49:43.750147 9338 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0920 05:49:43.750256 9338 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-3" } route-table:rtb-3 ok I0920 05:49:43.750358 9338 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0920 05:49:43.750438 9338 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0920 05:49:43.750710 9338 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateKopeio (0.40s) === RUN TestLifecycleIPv6 I0920 05:49:43.751020 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:43.751278 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:43.751328 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:43.751370 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:43.751418 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:43.751459 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:43.751490 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:43.751528 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:43.751554 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:43.751619 9338 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal-ipv6.example.com I0920 05:49:43.780358 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:43.784450 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:43.784550 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:43.822704 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:43.988819 9338 iamrole.go:89] ListRoles: { } I0920 05:49:43.990555 9338 executor.go:111] Tasks: 0 done / 106 total; 45 can run I0920 05:49:43.990783 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } I0920 05:49:43.990864 9338 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0920 05:49:43.990998 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:43.991158 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0920 05:49:43.992316 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0920 05:49:43.992532 9338 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0920 05:49:43.992652 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0920 05:49:43.992709 9338 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0920 05:49:43.992846 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0920 05:49:43.993241 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } I0920 05:49:43.993346 9338 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal-ipv6.example.com" }] } I0920 05:49:43.993402 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] }] } I0920 05:49:43.993956 9338 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0920 05:49:43.994005 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0920 05:49:43.994477 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:43.994660 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:43.995098 9338 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0920 05:49:43.995256 9338 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0920 05:49:43.995456 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0920 05:49:43.995607 9338 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0920 05:49:43.997386 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0920 05:49:43.997473 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "minimal-ipv6.example.com" }] }] } I0920 05:49:43.997682 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:43.998016 9338 executor.go:111] Tasks: 45 done / 106 total; 21 can run I0920 05:49:43.998209 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0920 05:49:43.998308 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0920 05:49:43.998411 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0920 05:49:43.998383 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0920 05:49:43.998488 9338 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0920 05:49:43.998612 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0920 05:49:43.998618 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0920 05:49:43.998762 9338 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0920 05:49:43.998744 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0920 05:49:43.998695 9338 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-minimal-ipv6-example--bne5ih", Port: 443, Protocol: "TCP", Tags: [{ Key: "Name", Value: "tcp-minimal-ipv6-example--bne5ih" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0920 05:49:43.998800 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0920 05:49:43.998880 9338 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0920 05:49:43.998971 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0920 05:49:43.999025 9338 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0920 05:49:43.999037 9338 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0920 05:49:43.999136 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0920 05:49:43.999192 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0920 05:49:43.999298 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public-us-test-1a"] }] } I0920 05:49:43.999382 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0920 05:49:43.999424 9338 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" } ] }], VpcId: "vpc-1" } I0920 05:49:43.999531 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0920 05:49:43.999579 9338 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" } ] } I0920 05:49:43.999667 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }], VpcId: "vpc-1" } I0920 05:49:43.999763 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0920 05:49:43.999811 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:43.999856 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:44.000013 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "api-elb.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }], VpcId: "vpc-1" } I0920 05:49:44.000145 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0920 05:49:44.000204 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:44.000243 9338 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] } I0920 05:49:44.000315 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0920 05:49:44.000400 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:44.000485 9338 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0920 05:49:44.000530 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0920 05:49:44.000585 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0920 05:49:44.000637 9338 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0920 05:49:44.000800 9338 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0920 05:49:44.003529 9338 executor.go:111] Tasks: 66 done / 106 total; 30 can run I0920 05:49:44.003695 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:44.003795 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:44.003905 9338 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0920 05:49:44.003877 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0920 05:49:44.004130 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0920 05:49:44.004356 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" }] }] } I0920 05:49:44.004534 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" }] }] } I0920 05:49:44.004750 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0920 05:49:44.004969 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0920 05:49:44.005076 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0920 05:49:44.005237 9338 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0920 05:49:44.005335 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0920 05:49:44.005578 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0920 05:49:44.005847 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:44.005825 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", Ipv6CidrBlock: "2001:db8:0:111::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-1" } I0920 05:49:44.006043 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:44.006141 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:44.006229 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-2382to4000-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0920 05:49:44.006392 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.006501 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.006560 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0920 05:49:44.006681 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.006760 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.006887 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0920 05:49:44.007077 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0920 05:49:44.007224 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0920 05:49:44.007358 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0920 05:49:44.007486 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:44.007542 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:44.007582 9338 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0920 05:49:44.007625 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0920 05:49:44.007802 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-4003to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0920 05:49:44.007975 9338 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0920 05:49:44.008033 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0920 05:49:44.008167 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-1to2379-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0920 05:49:44.008306 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-udp-1to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0920 05:49:44.008438 9338 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0920 05:49:44.008493 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0920 05:49:44.009136 9338 executor.go:111] Tasks: 96 done / 106 total; 5 can run I0920 05:49:44.009246 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0920 05:49:44.009321 9338 loadbalancers.go:34] PageSize not implemented I0920 05:49:44.009290 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:44.009464 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:44.009444 9338 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "dualstack", Name: "api-minimal-ipv6-example--jhj9te", SubnetMappings: [{ SubnetId: "subnet-1" }], Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "api.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }], Type: "network" } I0920 05:49:44.009536 9338 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:44.009563 9338 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1", Port: 443, Protocol: "TCP" } I0920 05:49:44.009629 9338 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0920 05:49:44.009634 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0920 05:49:44.009661 9338 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "false" },{ Key: "access_logs.s3.enabled", Value: "false" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0920 05:49:44.009713 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0920 05:49:44.009787 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0920 05:49:44.009840 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:44.009883 9338 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0920 05:49:44.009964 9338 executor.go:111] Tasks: 101 done / 106 total; 5 can run I0920 05:49:44.010112 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0920 05:49:44.010131 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:44.010213 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0920 05:49:44.010277 9338 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "64:ff9b::/96", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0920 05:49:44.010280 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0920 05:49:44.010423 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0920 05:49:44.010559 9338 executor.go:111] Tasks: 106 done / 106 total; 0 can run I0920 05:49:44.010612 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:44.010646 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:44.010696 9338 dns.go:238] Pre-creating DNS records I0920 05:49:44.010721 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:44.016599 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:44.020410 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.020507 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.057885 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:44.229547 9338 iamrole.go:89] ListRoles: { } I0920 05:49:44.231676 9338 executor.go:111] Tasks: 0 done / 106 total; 45 can run I0920 05:49:44.232000 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0920 05:49:44.232231 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0920 05:49:44.232258 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:44.232371 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:44.232472 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:44.232570 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0920 05:49:44.232846 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0920 05:49:44.232997 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0920 05:49:44.233032 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0920 05:49:44.233082 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:44.235037 9338 executor.go:111] Tasks: 45 done / 106 total; 21 can run I0920 05:49:44.235174 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:44.235263 9338 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0920 05:49:44.235362 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0920 05:49:44.235382 9338 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0920 05:49:44.235311 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0920 05:49:44.235436 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0920 05:49:44.235492 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0920 05:49:44.235649 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0920 05:49:44.235820 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:44.235873 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:44.236004 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0920 05:49:44.236195 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0920 05:49:44.236276 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0920 05:49:44.236418 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:44.236485 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0920 05:49:44.240683 9338 executor.go:111] Tasks: 66 done / 106 total; 30 can run I0920 05:49:44.240837 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.241022 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0920 05:49:44.241252 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:44.241503 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:44.241638 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:44.242055 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:44.242157 9338 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0920 05:49:44.242225 9338 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0920 05:49:44.242329 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:44.242429 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.242518 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:44.242618 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0920 05:49:44.242724 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.242892 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.244997 9338 executor.go:111] Tasks: 96 done / 106 total; 5 can run I0920 05:49:44.245128 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0920 05:49:44.245221 9338 loadbalancers.go:34] PageSize not implemented I0920 05:49:44.245130 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:44.245253 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0920 05:49:44.245334 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:44.245370 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0920 05:49:44.245415 9338 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0920 05:49:44.245431 9338 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0920 05:49:44.245477 9338 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0920 05:49:44.245515 9338 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0920 05:49:44.245650 9338 executor.go:111] Tasks: 101 done / 106 total; 5 can run I0920 05:49:44.245727 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:44.245869 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0920 05:49:44.245976 9338 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0920 05:49:44.246007 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0920 05:49:44.246034 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0920 05:49:44.246014 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0920 05:49:44.246161 9338 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0920 05:49:44.246207 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0920 05:49:44.246250 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0920 05:49:44.246394 9338 executor.go:111] Tasks: 106 done / 106 total; 0 can run I0920 05:49:44.249466 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0920 05:49:44.249539 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0920 05:49:44.249613 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0920 05:49:44.249677 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0920 05:49:44.249742 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0920 05:49:44.249813 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0920 05:49:44.249881 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0920 05:49:44.249942 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0920 05:49:44.250033 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0920 05:49:44.250106 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0920 05:49:44.250183 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0920 05:49:44.250266 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0920 05:49:44.250338 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0920 05:49:44.250411 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0920 05:49:44.250481 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0920 05:49:44.250537 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0920 05:49:44.250602 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0920 05:49:44.250670 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0920 05:49:44.250727 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0920 05:49:44.250797 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0920 05:49:44.250857 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0920 05:49:44.250925 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0920 05:49:44.250988 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:44.251063 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0920 05:49:44.251116 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0920 05:49:44.251178 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0920 05:49:44.251235 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0920 05:49:44.251302 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0920 05:49:44.251362 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0920 05:49:44.251424 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0920 05:49:44.253078 9338 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0920 05:49:44.253116 9338 keypairs.go:95] DescribeKeyPairs: { } I0920 05:49:44.253178 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0920 05:49:44.253261 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0920 05:49:44.253379 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0920 05:49:44.253472 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0920 05:49:44.253564 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0920 05:49:44.253642 9338 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0920 05:49:44.253704 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0920 05:49:44.253777 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0920 05:49:44.253859 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0920 05:49:44.253929 9338 routetable.go:53] DescribeRouteTables: { } I0920 05:49:44.253973 9338 natgateway.go:117] DescribeNatGateways: { } I0920 05:49:44.254039 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0920 05:49:44.254097 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } W0920 05:49:44.254184 9338 api.go:53] PageSize not implemented I0920 05:49:44.254214 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0920 05:49:44.254259 9338 loadbalancers.go:34] PageSize not implemented I0920 05:49:44.254280 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0920 05:49:44.254343 9338 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0920 05:49:44.254384 9338 targetgroups.go:35] PageSize not implemented I0920 05:49:44.254412 9338 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3" ] } I0920 05:49:44.254519 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:44.254567 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:44.254635 9338 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0920 05:49:44.254693 9338 iamrole.go:89] ListRoles: { } I0920 05:49:44.254754 9338 internetgateways.go:101] DescribeInternetGateways: { } I0920 05:49:44.254824 9338 routetable.go:53] DescribeRouteTables: { } I0920 05:49:44.254892 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I0920 05:49:44.254949 9338 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0920 05:49:44.255013 9338 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0920 05:49:44.255596 9338 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0920 05:49:44.255673 9338 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } iam-instance-profile:masters.minimal-ipv6.example.com ok I0920 05:49:44.255658 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-group:nodes.minimal-ipv6.example.com ok I0920 05:49:44.255751 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } autoscaling-group:master-us-test-1a.masters.minimal-ipv6.example.com ok I0920 05:49:44.255790 9338 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0920 05:49:44.255800 9338 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4 ok I0920 05:49:44.255851 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:44.255863 9338 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } security-group:sg-2 ok I0920 05:49:44.255942 9338 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } iam-instance-profile:nodes.minimal-ipv6.example.com ok I0920 05:49:44.255724 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [ { Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } } ] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/AAAA/kops-controller.internal.minimal-ipv6.example.com. ok I0920 05:49:44.255932 9338 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1 ok I0920 05:49:44.255936 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 } ] } W0920 05:49:44.256103 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:44.256126 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0920 05:49:44.256164 9338 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0920 05:49:44.256245 9338 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0920 05:49:44.256299 9338 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0920 05:49:44.256356 9338 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0920 05:49:44.256404 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0920 05:49:44.256457 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0920 05:49:44.256779 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:44.256800 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok autoscaling-config:lt-1 ok I0920 05:49:44.256844 9338 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0920 05:49:44.256907 9338 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-2 ok I0920 05:49:44.256997 9338 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal-ipv6.example.com" } I0920 05:49:44.257048 9338 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0920 05:49:44.257063 9338 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0920 05:49:44.257095 9338 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal-ipv6.example.com" } iam-role:nodes.minimal-ipv6.example.com ok I0920 05:49:44.257054 9338 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I0920 05:49:44.257126 9338 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal-ipv6.example.com" } I0920 05:49:44.257216 9338 iamrole.go:180] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0920 05:49:44.257231 9338 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0920 05:49:44.257264 9338 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal-ipv6.example.com" } iam-role:masters.minimal-ipv6.example.com ok I0920 05:49:44.257149 9338 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0920 05:49:44.257339 9338 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0920 05:49:44.257420 9338 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0920 05:49:44.257500 9338 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0920 05:49:44.257803 9338 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleIPv6 (0.51s) === RUN TestLifecycleSharedVPC I0920 05:49:44.258176 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:44.258437 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:44.258484 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:44.258529 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:44.258576 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:44.258621 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:44.258652 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:44.258690 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:44.258715 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:44.258778 9338 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedvpc.example.com I0920 05:49:44.288168 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:44.291420 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.291524 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.330041 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:44.424217 9338 iamrole.go:89] ListRoles: { } I0920 05:49:44.425416 9338 executor.go:111] Tasks: 0 done / 84 total; 42 can run I0920 05:49:44.426485 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0920 05:49:44.426746 9338 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0920 05:49:44.426886 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0920 05:49:44.427164 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:44.427655 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] }] } I0920 05:49:44.427952 9338 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0920 05:49:44.428186 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0920 05:49:44.428303 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0920 05:49:44.428366 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0920 05:49:44.428422 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0920 05:49:44.428906 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } I0920 05:49:44.429005 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:44.429031 9338 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] } I0920 05:49:44.429085 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:44.429182 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } I0920 05:49:44.429328 9338 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0920 05:49:44.430484 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0920 05:49:44.432456 9338 executor.go:111] Tasks: 42 done / 84 total; 17 can run I0920 05:49:44.432577 9338 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I0920 05:49:44.432630 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0920 05:49:44.432801 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0920 05:49:44.432815 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0920 05:49:44.432950 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0920 05:49:44.432931 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0920 05:49:44.433078 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0920 05:49:44.433144 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:44.433190 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0920 05:49:44.433192 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0920 05:49:44.433411 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0920 05:49:44.433496 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" } ] }], VpcId: "vpc-12345678" } I0920 05:49:44.433607 9338 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-12345678" } I0920 05:49:44.433639 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0920 05:49:44.433730 9338 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0920 05:49:44.433795 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0920 05:49:44.434041 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:44.434137 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:44.434197 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0920 05:49:44.434253 9338 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0920 05:49:44.434363 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0920 05:49:44.434453 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0920 05:49:44.434499 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:44.436321 9338 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0920 05:49:44.439710 9338 executor.go:111] Tasks: 59 done / 84 total; 21 can run I0920 05:49:44.439921 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0920 05:49:44.440148 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-udp-1to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0920 05:49:44.440296 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0920 05:49:44.440532 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0920 05:49:44.440675 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-1to2379-masters.sharedvpc.example.com" }] }] } I0920 05:49:44.440886 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0920 05:49:44.441117 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.441213 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:44.441293 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:44.441358 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:44.441424 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0920 05:49:44.441578 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.441730 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0920 05:49:44.441917 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0920 05:49:44.441985 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0920 05:49:44.442086 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0920 05:49:44.442481 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0920 05:49:44.442758 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.442961 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.443021 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0920 05:49:44.443233 9338 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0920 05:49:44.443381 9338 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0920 05:49:44.443457 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-4003to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0920 05:49:44.443648 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0920 05:49:44.443813 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-2382to4000-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0920 05:49:44.444329 9338 executor.go:111] Tasks: 80 done / 84 total; 2 can run I0920 05:49:44.444561 9338 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0920 05:49:44.444679 9338 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0920 05:49:44.444727 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:44.444776 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:44.444830 9338 dns.go:238] Pre-creating DNS records I0920 05:49:44.444860 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:44.451233 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:44.454497 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.454589 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.491786 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:44.587178 9338 iamrole.go:89] ListRoles: { } I0920 05:49:44.588790 9338 executor.go:111] Tasks: 0 done / 84 total; 42 can run I0920 05:49:44.589526 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0920 05:49:44.589651 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0920 05:49:44.589694 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0920 05:49:44.589686 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:44.589746 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0920 05:49:44.589969 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0920 05:49:44.590179 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:44.590251 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:44.591473 9338 executor.go:111] Tasks: 42 done / 84 total; 17 can run I0920 05:49:44.591659 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0920 05:49:44.591860 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0920 05:49:44.591808 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0920 05:49:44.591988 9338 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I0920 05:49:44.591963 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0920 05:49:44.592122 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:44.592168 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:44.592263 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0920 05:49:44.592418 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0920 05:49:44.592614 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:44.596775 9338 executor.go:111] Tasks: 59 done / 84 total; 21 can run I0920 05:49:44.596876 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:44.597009 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:44.597250 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.597432 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.597573 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.597765 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.597968 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:44.599918 9338 executor.go:111] Tasks: 80 done / 84 total; 2 can run I0920 05:49:44.600196 9338 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0920 05:49:44.600332 9338 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0920 05:49:44.603248 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0920 05:49:44.603337 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0920 05:49:44.603379 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0920 05:49:44.603417 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0920 05:49:44.603451 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0920 05:49:44.603484 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0920 05:49:44.603517 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0920 05:49:44.603548 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0920 05:49:44.603585 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0920 05:49:44.603623 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0920 05:49:44.603663 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0920 05:49:44.603695 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0920 05:49:44.603726 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0920 05:49:44.603764 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0920 05:49:44.603796 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0920 05:49:44.603830 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0920 05:49:44.603866 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:44.603904 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0920 05:49:44.603936 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0920 05:49:44.603968 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0920 05:49:44.604000 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0920 05:49:44.604034 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0920 05:49:44.604071 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0920 05:49:44.605503 9338 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0920 05:49:44.605534 9338 keypairs.go:95] DescribeKeyPairs: { } I0920 05:49:44.605574 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0920 05:49:44.605624 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0920 05:49:44.605679 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0920 05:49:44.605743 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0920 05:49:44.605786 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0920 05:49:44.605830 9338 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0920 05:49:44.605870 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0920 05:49:44.605909 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0920 05:49:44.605955 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0920 05:49:44.606014 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0920 05:49:44.606051 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } W0920 05:49:44.606100 9338 api.go:53] PageSize not implemented I0920 05:49:44.606124 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0920 05:49:44.606153 9338 loadbalancers.go:34] PageSize not implemented I0920 05:49:44.606173 9338 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0920 05:49:44.606198 9338 targetgroups.go:35] PageSize not implemented I0920 05:49:44.606216 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0920 05:49:44.606266 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:44.606297 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:44.606340 9338 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0920 05:49:44.606383 9338 iamrole.go:89] ListRoles: { } I0920 05:49:44.606426 9338 internetgateways.go:101] DescribeInternetGateways: { } I0920 05:49:44.606469 9338 routetable.go:53] DescribeRouteTables: { } I0920 05:49:44.606499 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:nodes.sharedvpc.example.com ok autoscaling-config:lt-1 ok I0920 05:49:44.606930 9338 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0920 05:49:44.607000 9338 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } iam-instance-profile:masters.sharedvpc.example.com ok I0920 05:49:44.606969 9338 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } autoscaling-group:master-us-test-1a.masters.sharedvpc.example.com ok volume:vol-1 ok I0920 05:49:44.607052 9338 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0920 05:49:44.607115 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:44.607112 9338 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0920 05:49:44.607193 9338 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } iam-instance-profile:nodes.sharedvpc.example.com ok I0920 05:49:44.607032 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.sharedvpc.example.com. ok I0920 05:49:44.607186 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0920 05:49:44.607401 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:44.607425 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0920 05:49:44.607461 9338 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-2 ok I0920 05:49:44.607545 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:44.607593 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0920 05:49:44.607701 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:44.607714 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0920 05:49:44.607810 9338 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedvpc.example.com" } I0920 05:49:44.607862 9338 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0920 05:49:44.607834 9338 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } I0920 05:49:44.607877 9338 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } subnet:subnet-1 ok I0920 05:49:44.607918 9338 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedvpc.example.com" } iam-role:nodes.sharedvpc.example.com ok I0920 05:49:44.607949 9338 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedvpc.example.com" } I0920 05:49:44.607990 9338 iamrole.go:180] ListAttachedRolePolicies: masters.sharedvpc.example.com I0920 05:49:44.608002 9338 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0920 05:49:44.608029 9338 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedvpc.example.com" } iam-role:masters.sharedvpc.example.com ok I0920 05:49:44.608080 9338 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0920 05:49:44.608350 9338 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedVPC (0.35s) === RUN TestLifecycleComplex I0920 05:49:44.608634 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:44.608867 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:44.608912 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:44.608953 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:44.609003 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:44.609046 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:44.609075 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:44.609111 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:44.609136 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:44.609198 9338 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster complex.example.com I0920 05:49:44.644589 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0920 05:49:44.649871 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.650033 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.690642 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:44.878374 9338 iamrole.go:89] ListRoles: { } I0920 05:49:44.880758 9338 executor.go:111] Tasks: 0 done / 122 total; 51 can run W0920 05:49:44.880977 9338 api.go:53] PageSize not implemented I0920 05:49:44.881334 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0920 05:49:44.881409 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:44.881522 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0920 05:49:44.881334 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } I0920 05:49:44.881644 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0920 05:49:44.881680 9338 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", Tags: [ { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] } I0920 05:49:44.881813 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:44.881867 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } I0920 05:49:44.881817 9338 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0920 05:49:44.881969 9338 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] } I0920 05:49:44.882010 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:44.882171 9338 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "a.etcd-events.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/events", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0920 05:49:44.882282 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0920 05:49:44.882319 9338 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "complex.example.com" } ] }] } I0920 05:49:44.882405 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0920 05:49:44.882434 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0920 05:49:44.882500 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0920 05:49:44.882558 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:44.882662 9338 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-main.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/main", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0920 05:49:44.882757 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0920 05:49:44.882803 9338 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0920 05:49:44.882900 9338 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0920 05:49:44.882959 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0920 05:49:44.883051 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:44.884004 9338 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0920 05:49:44.884177 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0920 05:49:44.884267 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0920 05:49:44.884418 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0920 05:49:44.884393 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:44.884493 9338 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0920 05:49:44.884614 9338 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0920 05:49:44.884690 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:44.884723 9338 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0920 05:49:44.884755 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0920 05:49:44.884748 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.96.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-east-1a-utility.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0920 05:49:44.884919 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:44.884990 9338 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-complex-example-com-vpjolq", Port: 443, Protocol: "TCP", Tags: [ { Key: "Name", Value: "tcp-complex-example-com-vpjolq" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0920 05:49:44.885113 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I0920 05:49:44.885228 9338 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0920 05:49:44.885207 9338 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tls-complex-example-com-5nursn", Port: 443, Protocol: "TLS", Tags: [ { Key: "Name", Value: "tls-complex-example-com-5nursn" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0920 05:49:44.885334 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:44.885356 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-east-1a-private.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0920 05:49:44.885486 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0920 05:49:44.885532 9338 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.1.0.0/16", VpcId: "vpc-1" } I0920 05:49:44.885594 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0920 05:49:44.885647 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:44.885837 9338 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0920 05:49:44.885972 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:44.886053 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:44.886145 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0920 05:49:44.886274 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I0920 05:49:44.886254 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0920 05:49:44.886378 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0920 05:49:44.886482 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:44.886570 9338 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.2.0.0/16", VpcId: "vpc-1" } I0920 05:49:44.886619 9338 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0920 05:49:44.886673 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:44.886770 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0920 05:49:44.886826 9338 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0920 05:49:44.886889 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0920 05:49:44.886945 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0920 05:49:44.886985 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0920 05:49:44.887036 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:44.887077 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "masters.complex.example.com" } ] }], VpcId: "vpc-1" } I0920 05:49:44.887186 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0920 05:49:44.887241 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:44.887282 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Name", Value: "api-elb.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0920 05:49:44.886976 9338 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0920 05:49:44.887382 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0920 05:49:44.887425 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0920 05:49:44.887465 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0920 05:49:44.887599 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:44.887597 9338 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0920 05:49:44.887695 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:44.887744 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0920 05:49:44.887794 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0920 05:49:44.887930 9338 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I0920 05:49:44.887935 9338 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] } I0920 05:49:44.888052 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0920 05:49:44.888103 9338 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] } I0920 05:49:44.889628 9338 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0920 05:49:44.889806 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0920 05:49:44.889889 9338 loadbalancers.go:34] PageSize not implemented I0920 05:49:44.889815 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:44.889963 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:44.890032 9338 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "api-complex-example-com-vd3t5n", SubnetMappings: [{ AllocationId: "eipalloc-012345a678b9cdefa", SubnetId: "subnet-3" }], Tags: [ { Key: "Name", Value: "api.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ], Type: "network" } I0920 05:49:44.890179 9338 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-test-1:000000000000:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0920 05:49:44.890088 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0920 05:49:44.890300 9338 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0920 05:49:44.890378 9338 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [ { Key: "load_balancing.cross_zone.enabled", Value: "true" }, { Key: "access_logs.s3.enabled", Value: "true" }, { Key: "access_logs.s3.bucket", Value: "access-log-example" }, { Key: "access_logs.s3.prefix", Value: "" } ], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0920 05:49:44.890418 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-4003to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }] } I0920 05:49:44.890662 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }] } I0920 05:49:44.890769 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0920 05:49:44.890936 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0920 05:49:44.891106 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:44.891181 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:44.891254 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:44.891328 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:44.891416 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:44.891486 9338 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0920 05:49:44.891537 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0920 05:49:44.891667 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0920 05:49:44.891709 9338 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0920 05:49:44.891758 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-2382to4000-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }] } I0920 05:49:44.892019 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0920 05:49:44.892072 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0920 05:49:44.892114 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0920 05:49:44.892268 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0920 05:49:44.892309 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0920 05:49:44.892349 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0920 05:49:44.892513 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0920 05:49:44.892667 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-masters.complex.example.com" } ] }] } I0920 05:49:44.892822 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }] } I0920 05:49:44.892918 9338 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", RouteTableId: "rtb-2", TransitGatewayId: "tgw-123456" } I0920 05:49:44.892973 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.0/24-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0920 05:49:44.893131 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0920 05:49:44.893232 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.complex.example.com" } ] }] } I0920 05:49:44.893415 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0920 05:49:44.893563 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }] } I0920 05:49:44.893656 9338 routetable.go:149] CreateRoute: { DestinationCidrBlock: "192.168.1.10/32", RouteTableId: "rtb-2", TransitGatewayId: "tgw-0123456" } I0920 05:49:44.893708 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.893814 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.893938 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }] } I0920 05:49:44.894161 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.894277 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.894417 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-::/0" } ] }] } I0920 05:49:44.894635 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-1to2379-masters.complex.example.com" } ] }] } I0920 05:49:44.894812 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }] } I0920 05:49:44.894918 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-3" } I0920 05:49:44.894963 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0920 05:49:44.895060 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0920 05:49:44.895160 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }] } I0920 05:49:44.895303 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0920 05:49:44.895449 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-udp-1to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }] } I0920 05:49:44.895597 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }] } I0920 05:49:44.896621 9338 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0920 05:49:44.896719 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0920 05:49:44.896878 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0920 05:49:44.896995 9338 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0920 05:49:44.897110 9338 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0920 05:49:44.897162 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:44.897198 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:44.897252 9338 dns.go:238] Pre-creating DNS records I0920 05:49:44.897278 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:44.905158 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0920 05:49:44.909734 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.909843 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:44.948850 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:45.138799 9338 iamrole.go:89] ListRoles: { } I0920 05:49:45.140952 9338 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0920 05:49:45.141217 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.141358 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0920 05:49:45.141390 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0920 05:49:45.141448 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } W0920 05:49:45.141522 9338 api.go:53] PageSize not implemented I0920 05:49:45.141491 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0920 05:49:45.141659 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.141698 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:45.141822 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.142065 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0920 05:49:45.142192 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.142335 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0920 05:49:45.142442 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0920 05:49:45.145455 9338 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0920 05:49:45.145737 9338 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0920 05:49:45.145885 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0920 05:49:45.145725 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.146182 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0920 05:49:45.146578 9338 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0920 05:49:45.146712 9338 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0920 05:49:45.146360 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.146860 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0920 05:49:45.147029 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:45.147238 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.147999 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0920 05:49:45.148071 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.148220 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:45.148312 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:45.148391 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:45.148469 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0920 05:49:45.149296 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0920 05:49:45.149382 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0920 05:49:45.149551 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.149732 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.149913 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0920 05:49:45.150213 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:45.150335 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:45.150487 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0920 05:49:45.150573 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:45.153676 9338 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0920 05:49:45.153949 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:45.154079 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0920 05:49:45.154176 9338 loadbalancers.go:34] PageSize not implemented I0920 05:49:45.154196 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:45.154148 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:45.154248 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:45.154295 9338 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0920 05:49:45.154337 9338 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0920 05:49:45.154374 9338 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0920 05:49:45.154345 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:45.154407 9338 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0920 05:49:45.154558 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:45.154749 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:45.154887 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:45.155163 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:45.155323 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:45.155435 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:45.155593 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:45.155705 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:45.158822 9338 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0920 05:49:45.158954 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0920 05:49:45.159046 9338 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0920 05:49:45.159071 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0920 05:49:45.159096 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:45.159175 9338 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0920 05:49:45.159327 9338 executor.go:111] Tasks: 122 done / 122 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.sslCertificate=arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678] I0920 05:49:45.166447 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:45.170144 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.182133 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0920 05:49:45.186555 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:45.186673 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:45.225331 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:45.419295 9338 iamrole.go:89] ListRoles: { } I0920 05:49:45.421494 9338 executor.go:111] Tasks: 0 done / 122 total; 51 can run W0920 05:49:45.422014 9338 api.go:53] PageSize not implemented I0920 05:49:45.421896 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0920 05:49:45.422124 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:45.422103 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0920 05:49:45.422310 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.422519 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0920 05:49:45.422646 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.422758 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0920 05:49:45.422789 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0920 05:49:45.422843 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:45.422880 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.423006 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0920 05:49:45.423105 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0920 05:49:45.424726 9338 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0920 05:49:45.424899 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.425007 9338 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0920 05:49:45.425100 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0920 05:49:45.425025 9338 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0920 05:49:45.425229 9338 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0920 05:49:45.425306 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0920 05:49:45.425305 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0920 05:49:45.425105 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.425447 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:45.425591 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:45.425689 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:45.425748 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0920 05:49:45.425927 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0920 05:49:45.425984 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I0920 05:49:45.426134 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0920 05:49:45.426186 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0920 05:49:45.426391 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:45.426465 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:45.426566 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.426733 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.426868 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0920 05:49:45.427039 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:45.427082 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:45.427156 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.427321 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0920 05:49:45.430253 9338 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0920 05:49:45.430419 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:45.430561 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0920 05:49:45.430661 9338 loadbalancers.go:34] PageSize not implemented I0920 05:49:45.430624 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:45.430694 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:45.430750 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:45.430789 9338 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0920 05:49:45.430831 9338 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0920 05:49:45.430795 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:45.430868 9338 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0920 05:49:45.430918 9338 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0920 05:49:45.431009 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:45.431110 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { Names: ["api-complex-example-com-vd3t5n"] } I0920 05:49:45.431104 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:45.431149 9338 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0920 05:49:45.431179 9338 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/1" } I0920 05:49:45.431214 9338 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/2" } I0920 05:49:45.431248 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:45.431246 9338 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0920 05:49:45.431404 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:45.431444 9338 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0920 05:49:45.431519 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:45.432369 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:45.433237 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:45.433506 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:45.433715 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:45.433948 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:45.439100 9338 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0920 05:49:45.439284 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0920 05:49:45.439399 9338 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0920 05:49:45.439426 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0920 05:49:45.439452 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:45.439554 9338 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0920 05:49:45.439704 9338 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0920 05:49:45.439757 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:45.439794 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:45.439849 9338 dns.go:238] Pre-creating DNS records I0920 05:49:45.448516 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0920 05:49:45.453656 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:45.453759 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:45.491620 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:45.674345 9338 iamrole.go:89] ListRoles: { } I0920 05:49:45.676638 9338 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0920 05:49:45.676848 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.677079 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0920 05:49:45.677206 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.677319 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0920 05:49:45.677379 9338 api.go:53] PageSize not implemented I0920 05:49:45.677349 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } I0920 05:49:45.677512 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.677598 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0920 05:49:45.677621 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0920 05:49:45.677652 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0920 05:49:45.677820 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0920 05:49:45.677925 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0920 05:49:45.678022 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:45.679317 9338 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0920 05:49:45.679466 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0920 05:49:45.679515 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0920 05:49:45.679624 9338 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0920 05:49:45.679661 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.679724 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0920 05:49:45.679792 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:45.679810 9338 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0920 05:49:45.679842 9338 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0920 05:49:45.679926 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0920 05:49:45.679899 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.680106 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0920 05:49:45.680156 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.680292 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:45.680384 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:45.680466 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.680598 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0920 05:49:45.680651 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0920 05:49:45.680856 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0920 05:49:45.680913 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:45.680970 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0920 05:49:45.681138 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:45.681233 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.681439 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.681574 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:45.681664 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:45.684805 9338 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0920 05:49:45.685030 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:45.685173 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:45.685288 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } I0920 05:49:45.685309 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0920 05:49:45.685373 9338 loadbalancers.go:34] PageSize not implemented I0920 05:49:45.685443 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:45.685511 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:45.685553 9338 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0920 05:49:45.685504 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:45.685594 9338 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0920 05:49:45.685632 9338 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0920 05:49:45.685666 9338 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0920 05:49:45.685700 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:45.685851 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:45.686046 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:45.686184 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:45.686309 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:45.686554 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:45.686742 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:45.688381 9338 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0920 05:49:45.688461 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0920 05:49:45.688551 9338 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0920 05:49:45.688581 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0920 05:49:45.688609 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:45.688687 9338 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0920 05:49:45.688805 9338 executor.go:111] Tasks: 122 done / 122 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.additionalSecurityGroups=sg-123456] I0920 05:49:45.692538 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:45.696664 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.708877 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0920 05:49:45.713345 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:45.713450 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:45.750958 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:45.941738 9338 iamrole.go:89] ListRoles: { } I0920 05:49:45.944057 9338 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0920 05:49:45.944376 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.944679 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } W0920 05:49:45.944912 9338 api.go:53] PageSize not implemented I0920 05:49:45.944904 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:45.944901 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.945124 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0920 05:49:45.945299 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0920 05:49:45.945843 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0920 05:49:45.946011 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.946248 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.946409 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0920 05:49:45.946453 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0920 05:49:45.946501 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0920 05:49:45.946620 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:45.947748 9338 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0920 05:49:45.947935 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0920 05:49:45.947889 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.948095 9338 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0920 05:49:45.948087 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0920 05:49:45.948230 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0920 05:49:45.948194 9338 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0920 05:49:45.948290 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.948337 9338 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0920 05:49:45.948443 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0920 05:49:45.948461 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:45.948575 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:45.948655 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.948806 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:45.948902 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.949060 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0920 05:49:45.949103 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:45.949232 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0920 05:49:45.949272 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:45.949357 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] }] } I0920 05:49:45.949494 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:45.949550 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0920 05:49:45.949764 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:45.949860 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:45.950018 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0920 05:49:45.950199 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0920 05:49:45.954560 9338 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0920 05:49:45.954741 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:45.954906 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:45.954956 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0920 05:49:45.955066 9338 loadbalancers.go:34] PageSize not implemented I0920 05:49:45.955030 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:45.955093 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:45.955150 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:45.955193 9338 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0920 05:49:45.955168 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:45.955234 9338 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0920 05:49:45.955282 9338 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0920 05:49:45.955315 9338 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0920 05:49:45.955316 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:45.956008 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:45.956118 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:45.956196 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:45.956301 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:45.956426 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:45.956636 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:45.962340 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:45.962439 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:45.963387 9338 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0920 05:49:45.963482 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0920 05:49:45.963574 9338 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0920 05:49:45.963603 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0920 05:49:45.963631 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:45.963758 9338 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0920 05:49:45.963961 9338 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0920 05:49:45.964015 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:45.964052 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:45.964106 9338 dns.go:238] Pre-creating DNS records I0920 05:49:45.972848 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0920 05:49:45.977508 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:45.977629 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.015727 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:46.188438 9338 iamrole.go:89] ListRoles: { } I0920 05:49:46.190228 9338 executor.go:111] Tasks: 0 done / 123 total; 52 can run W0920 05:49:46.190419 9338 api.go:53] PageSize not implemented I0920 05:49:46.190488 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0920 05:49:46.192077 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:46.192457 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0920 05:49:46.192610 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0920 05:49:46.192745 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.192946 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.193131 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0920 05:49:46.193264 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } I0920 05:49:46.193737 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0920 05:49:46.193975 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0920 05:49:46.194138 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0920 05:49:46.194208 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0920 05:49:46.194367 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:46.197850 9338 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0920 05:49:46.198006 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0920 05:49:46.198066 9338 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0920 05:49:46.197996 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0920 05:49:46.198176 9338 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0920 05:49:46.198192 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0920 05:49:46.198325 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0920 05:49:46.198401 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:46.198414 9338 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0920 05:49:46.198521 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0920 05:49:46.198529 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0920 05:49:46.198753 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0920 05:49:46.198816 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.198969 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.199132 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:46.199174 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.199310 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.199462 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0920 05:49:46.199503 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.199637 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0920 05:49:46.199683 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.199874 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:46.199998 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:46.200084 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:46.200159 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:46.200232 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:46.203448 9338 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0920 05:49:46.203703 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } I0920 05:49:46.203699 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0920 05:49:46.203771 9338 loadbalancers.go:34] PageSize not implemented I0920 05:49:46.203913 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:46.203961 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:46.204005 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:46.204042 9338 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0920 05:49:46.204088 9338 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0920 05:49:46.204084 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:46.204120 9338 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0920 05:49:46.204215 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:46.204303 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:46.204242 9338 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0920 05:49:46.204489 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:46.204648 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.204831 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.205021 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.205230 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:46.205615 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.207795 9338 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0920 05:49:46.207984 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0920 05:49:46.208139 9338 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0920 05:49:46.208189 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0920 05:49:46.208238 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:46.208401 9338 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0920 05:49:46.208631 9338 executor.go:111] Tasks: 123 done / 123 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t3.large] I0920 05:49:46.214033 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:46.217692 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.217797 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.227541 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0920 05:49:46.232064 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.232167 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.232230 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.269954 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:46.461695 9338 iamrole.go:89] ListRoles: { } I0920 05:49:46.464137 9338 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0920 05:49:46.464537 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0920 05:49:46.464725 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0920 05:49:46.464875 9338 api.go:53] PageSize not implemented I0920 05:49:46.464854 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:46.464863 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.465083 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0920 05:49:46.465229 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0920 05:49:46.465333 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.465461 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0920 05:49:46.465494 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0920 05:49:46.465555 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0920 05:49:46.465796 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0920 05:49:46.465976 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.466240 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:46.467392 9338 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0920 05:49:46.467587 9338 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0920 05:49:46.467561 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:46.467693 9338 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0920 05:49:46.467786 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0920 05:49:46.467706 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.467805 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0920 05:49:46.467879 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.468063 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0920 05:49:46.468003 9338 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0920 05:49:46.468156 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0920 05:49:46.468113 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0920 05:49:46.468294 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0920 05:49:46.468337 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0920 05:49:46.468537 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:46.468620 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.468811 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.468954 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0920 05:49:46.469110 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:46.469167 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.469278 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:46.469368 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.469468 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:46.469531 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:46.469616 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:46.469654 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0920 05:49:46.472933 9338 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0920 05:49:46.473176 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:46.473205 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0920 05:49:46.473336 9338 loadbalancers.go:34] PageSize not implemented I0920 05:49:46.473293 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:46.473372 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:46.473451 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:46.473530 9338 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0920 05:49:46.473591 9338 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0920 05:49:46.473650 9338 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0920 05:49:46.473574 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.473710 9338 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0920 05:49:46.473817 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.474118 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:46.474312 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.474445 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:46.474532 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:46.474598 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:46.474755 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:46.474820 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.478410 9338 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0920 05:49:46.478519 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0920 05:49:46.478622 9338 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0920 05:49:46.478650 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0920 05:49:46.478681 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:46.478968 9338 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0920 05:49:46.479262 9338 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0920 05:49:46.479381 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:46.479580 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:46.479756 9338 dns.go:238] Pre-creating DNS records I0920 05:49:46.489994 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0920 05:49:46.494914 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.495021 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.495103 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.533627 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:46.722665 9338 iamrole.go:89] ListRoles: { } I0920 05:49:46.726399 9338 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0920 05:49:46.726863 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0920 05:49:46.727034 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } W0920 05:49:46.727151 9338 api.go:53] PageSize not implemented I0920 05:49:46.727192 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.727311 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:46.727325 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.727602 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0920 05:49:46.727784 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.727966 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0920 05:49:46.728018 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0920 05:49:46.728063 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0920 05:49:46.728178 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.728431 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0920 05:49:46.728602 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:46.729513 9338 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0920 05:49:46.729690 9338 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0920 05:49:46.729710 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0920 05:49:46.729826 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0920 05:49:46.729906 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.730040 9338 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0920 05:49:46.729849 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0920 05:49:46.730125 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0920 05:49:46.730114 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:46.730168 9338 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0920 05:49:46.730213 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.730388 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0920 05:49:46.730443 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.730586 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:46.730637 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.730779 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0920 05:49:46.730964 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:46.731026 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0920 05:49:46.731075 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:46.731161 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:46.731245 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0920 05:49:46.731389 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:46.731445 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:46.731600 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0920 05:49:46.731745 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.731897 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0920 05:49:46.735654 9338 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0920 05:49:46.735784 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:46.735883 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:46.735943 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0920 05:49:46.736049 9338 loadbalancers.go:34] PageSize not implemented I0920 05:49:46.736076 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:46.736144 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:46.736141 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:46.736202 9338 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0920 05:49:46.736349 9338 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0920 05:49:46.736313 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.736389 9338 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0920 05:49:46.736420 9338 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0920 05:49:46.736417 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:46.736556 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.736785 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:46.737191 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:46.737404 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:46.737523 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.737662 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.740482 9338 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0920 05:49:46.740669 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0920 05:49:46.740772 9338 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0920 05:49:46.740796 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0920 05:49:46.740819 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:46.740921 9338 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0920 05:49:46.741071 9338 executor.go:111] Tasks: 123 done / 123 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t2.medium] I0920 05:49:46.744274 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:46.747874 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.747966 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.748111 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.757783 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0920 05:49:46.762226 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.762321 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.762383 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.762437 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.800769 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:46.982268 9338 iamrole.go:89] ListRoles: { } I0920 05:49:46.984582 9338 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0920 05:49:46.984945 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0920 05:49:46.985066 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0920 05:49:46.985239 9338 api.go:53] PageSize not implemented I0920 05:49:46.985253 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.985498 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.985658 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0920 05:49:46.985758 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.985872 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0920 05:49:46.985941 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0920 05:49:46.986001 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.986077 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0920 05:49:46.986098 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0920 05:49:46.986139 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:46.986167 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0920 05:49:46.987591 9338 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0920 05:49:46.987732 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:46.987847 9338 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0920 05:49:46.987894 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0920 05:49:46.987900 9338 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0920 05:49:46.988000 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0920 05:49:46.987897 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:46.988100 9338 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0920 05:49:46.988114 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0920 05:49:46.988169 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0920 05:49:46.988213 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0920 05:49:46.988438 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:46.988502 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:46.988602 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0920 05:49:46.988746 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:46.988820 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.988966 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.989127 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0920 05:49:46.989173 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.989280 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0920 05:49:46.989412 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:46.989518 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] }] } I0920 05:49:46.989633 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:46.989726 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0920 05:49:46.989787 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0920 05:49:46.989864 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:46.994393 9338 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0920 05:49:46.994698 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:46.994803 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } I0920 05:49:46.994869 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0920 05:49:46.994935 9338 loadbalancers.go:34] PageSize not implemented I0920 05:49:46.995014 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:46.995051 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:46.995095 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:46.995130 9338 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0920 05:49:46.995170 9338 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0920 05:49:46.995203 9338 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0920 05:49:46.995241 9338 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0920 05:49:46.995177 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:46.995546 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:46.995656 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.995844 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.996007 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.996188 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:46.996353 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:46.996532 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:46.999185 9338 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0920 05:49:46.999275 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0920 05:49:46.999358 9338 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0920 05:49:46.999381 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0920 05:49:46.999406 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:46.999538 9338 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0920 05:49:46.999693 9338 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0920 05:49:46.999745 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:46.999781 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:46.999838 9338 dns.go:238] Pre-creating DNS records I0920 05:49:47.007777 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0920 05:49:47.012319 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.012418 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.012481 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.012536 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.050574 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:47.241761 9338 iamrole.go:89] ListRoles: { } I0920 05:49:47.243493 9338 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0920 05:49:47.243809 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0920 05:49:47.243897 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:47.243921 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:47.244018 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } W0920 05:49:47.244104 9338 api.go:53] PageSize not implemented I0920 05:49:47.244102 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0920 05:49:47.244186 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0920 05:49:47.244431 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:47.244696 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:47.244932 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:47.245118 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0920 05:49:47.245170 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0920 05:49:47.245265 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0920 05:49:47.245477 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:47.246377 9338 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0920 05:49:47.246506 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:47.246567 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0920 05:49:47.246675 9338 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0920 05:49:47.246755 9338 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0920 05:49:47.246757 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0920 05:49:47.246636 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:47.246841 9338 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0920 05:49:47.246911 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0920 05:49:47.246904 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0920 05:49:47.247122 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:47.248518 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0920 05:49:47.248902 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:47.249190 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:47.249422 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:47.249787 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0920 05:49:47.250000 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:47.250284 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:47.250519 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:47.251587 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0920 05:49:47.251724 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:47.251831 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:47.251992 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0920 05:49:47.252693 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0920 05:49:47.252778 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:47.252838 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:47.255256 9338 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0920 05:49:47.255460 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:47.255573 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0920 05:49:47.255654 9338 loadbalancers.go:34] PageSize not implemented I0920 05:49:47.255678 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:47.255726 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:47.255672 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:47.255761 9338 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0920 05:49:47.255809 9338 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0920 05:49:47.255847 9338 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0920 05:49:47.255819 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:47.255878 9338 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0920 05:49:47.256054 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:47.256154 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:47.256586 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.256763 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:47.256856 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:47.257030 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.257231 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.257416 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.260310 9338 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0920 05:49:47.260397 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0920 05:49:47.260509 9338 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0920 05:49:47.260542 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0920 05:49:47.260575 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:47.260705 9338 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0920 05:49:47.260867 9338 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0920 05:49:47.261266 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0920 05:49:47.261343 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0920 05:49:47.261386 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0920 05:49:47.261435 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0920 05:49:47.261479 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0920 05:49:47.261545 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0920 05:49:47.261592 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0920 05:49:47.261640 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0920 05:49:47.261688 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0920 05:49:47.261735 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0920 05:49:47.261797 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0920 05:49:47.261850 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0920 05:49:47.261908 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0920 05:49:47.261955 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0920 05:49:47.262007 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0920 05:49:47.262078 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0920 05:49:47.262187 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0920 05:49:47.262274 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0920 05:49:47.262362 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0920 05:49:47.262460 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:47.262556 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0920 05:49:47.262650 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0920 05:49:47.262735 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0920 05:49:47.262857 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0920 05:49:47.262959 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0920 05:49:47.263011 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0920 05:49:47.263078 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0920 05:49:47.263125 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0920 05:49:47.263182 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0920 05:49:47.265989 9338 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0920 05:49:47.266030 9338 keypairs.go:95] DescribeKeyPairs: { } I0920 05:49:47.266078 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:47.266153 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0920 05:49:47.266245 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:47.266326 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:47.266393 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:47.266473 9338 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:47.266528 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:47.266585 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0920 05:49:47.266686 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:47.266785 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0920 05:49:47.266843 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } W0920 05:49:47.266911 9338 api.go:53] PageSize not implemented I0920 05:49:47.266942 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0920 05:49:47.266974 9338 loadbalancers.go:34] PageSize not implemented I0920 05:49:47.266988 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0920 05:49:47.267043 9338 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0920 05:49:47.267076 9338 targetgroups.go:35] PageSize not implemented I0920 05:49:47.267097 9338 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5" ] } I0920 05:49:47.267195 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:47.267228 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:47.267282 9338 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0920 05:49:47.267325 9338 iamrole.go:89] ListRoles: { } I0920 05:49:47.267382 9338 internetgateways.go:101] DescribeInternetGateways: { } I0920 05:49:47.267450 9338 routetable.go:53] DescribeRouteTables: { } I0920 05:49:47.267516 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0920 05:49:47.268099 9338 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1 ok autoscaling-group:nodes.complex.example.com ok I0920 05:49:47.268180 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:master-us-test-1a.masters.complex.example.com ok I0920 05:49:47.268263 9338 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0920 05:49:47.268199 9338 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5" } I0920 05:49:47.268354 9338 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5 ok iam-instance-profile:masters.complex.example.com ok I0920 05:49:47.268402 9338 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0920 05:49:47.268272 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 } ] } W0920 05:49:47.268500 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:47.268494 9338 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } iam-instance-profile:nodes.complex.example.com ok I0920 05:49:47.268432 9338 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4 ok I0920 05:49:47.268348 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.complex.example.com. ok I0920 05:49:47.268526 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0920 05:49:47.268654 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0920 05:49:47.268719 9338 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0920 05:49:47.268797 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0920 05:49:47.268833 9338 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0920 05:49:47.268892 9338 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-1 ok I0920 05:49:47.268950 9338 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0920 05:49:47.269005 9338 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok autoscaling-config:lt-2 ok I0920 05:49:47.269074 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:47.269125 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 } ] } W0920 05:49:47.269473 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:47.269493 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0920 05:49:47.269524 9338 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0920 05:49:47.269624 9338 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I0920 05:49:47.269674 9338 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.complex.example.com" } I0920 05:49:47.269678 9338 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0920 05:49:47.269767 9338 iamrole.go:180] ListAttachedRolePolicies: nodes.complex.example.com I0920 05:49:47.269822 9338 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0920 05:49:47.269867 9338 iamrole.go:137] DeleteRole: { RoleName: "nodes.complex.example.com" } iam-role:nodes.complex.example.com ok I0920 05:49:47.269915 9338 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.complex.example.com" } I0920 05:49:47.270018 9338 iamrole.go:180] ListAttachedRolePolicies: masters.complex.example.com I0920 05:49:47.270034 9338 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0920 05:49:47.270067 9338 iamrole.go:137] DeleteRole: { RoleName: "masters.complex.example.com" } iam-role:masters.complex.example.com ok I0920 05:49:47.270134 9338 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0920 05:49:47.270217 9338 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0920 05:49:47.270296 9338 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0920 05:49:47.270595 9338 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleComplex (2.66s) === RUN TestLifecycleExternalLB I0920 05:49:47.270873 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:47.271111 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:47.271158 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:47.271197 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:47.271242 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:47.271285 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:47.271316 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:47.271354 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:47.271380 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:47.271445 9338 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster externallb.example.com I0920 05:49:47.295954 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:47.299234 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.299337 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.337535 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:47.435210 9338 iamrole.go:89] ListRoles: { } I0920 05:49:47.436956 9338 executor.go:111] Tasks: 0 done / 94 total; 50 can run I0920 05:49:47.437159 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } W0920 05:49:47.437396 9338 api.go:53] PageSize not implemented I0920 05:49:47.437392 9338 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } W0920 05:49:47.437558 9338 api.go:53] PageSize not implemented I0920 05:49:47.437487 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:47.437527 9338 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0920 05:49:47.437666 9338 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0920 05:49:47.437731 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } W0920 05:49:47.437783 9338 api.go:53] PageSize not implemented I0920 05:49:47.437797 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:47.437749 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } I0920 05:49:47.437877 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:47.437877 9338 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I0920 05:49:47.437583 9338 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0920 05:49:47.437998 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0920 05:49:47.437988 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:47.438101 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } I0920 05:49:47.438098 9338 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0920 05:49:47.438245 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0920 05:49:47.438218 9338 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] } I0920 05:49:47.438127 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0920 05:49:47.438309 9338 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0920 05:49:47.438482 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0920 05:49:47.438469 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0920 05:49:47.438537 9338 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0920 05:49:47.438707 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0920 05:49:47.438680 9338 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0920 05:49:47.438819 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0920 05:49:47.438864 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0920 05:49:47.438988 9338 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0920 05:49:47.439080 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0920 05:49:47.439119 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0920 05:49:47.439297 9338 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0920 05:49:47.439430 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0920 05:49:47.440157 9338 executor.go:111] Tasks: 50 done / 94 total; 19 can run I0920 05:49:47.440288 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:47.440480 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I0920 05:49:47.440514 9338 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I0920 05:49:47.440457 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0920 05:49:47.440616 9338 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0920 05:49:47.440614 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0920 05:49:47.440687 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0920 05:49:47.440727 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:47.440862 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0920 05:49:47.440836 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0920 05:49:47.440939 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0920 05:49:47.441066 9338 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0920 05:49:47.441026 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0920 05:49:47.440991 9338 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0920 05:49:47.441284 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0920 05:49:47.441399 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0920 05:49:47.441459 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:47.441506 9338 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0920 05:49:47.441554 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0920 05:49:47.441752 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" } ] }], VpcId: "vpc-1" } I0920 05:49:47.441757 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0920 05:49:47.442051 9338 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0920 05:49:47.442113 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.externallb.example.com" }] }], VpcId: "vpc-1" } I0920 05:49:47.442291 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0920 05:49:47.442387 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:47.442474 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0920 05:49:47.442573 9338 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0920 05:49:47.442747 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:47.442847 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:47.442659 9338 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0920 05:49:47.445357 9338 executor.go:111] Tasks: 69 done / 94 total; 21 can run I0920 05:49:47.445888 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:47.446037 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:47.446208 9338 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0920 05:49:47.446373 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.446472 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-udp-1to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0920 05:49:47.446686 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.446965 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0920 05:49:47.447147 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-4003to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0920 05:49:47.447302 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.447396 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:47.447477 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.externallb.example.com" }] }] } I0920 05:49:47.447622 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-1to2379-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0920 05:49:47.447764 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0920 05:49:47.447807 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0920 05:49:47.447939 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-2382to4000-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0920 05:49:47.448079 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.externallb.example.com" }] }] } I0920 05:49:47.448224 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0920 05:49:47.448365 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.448511 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0920 05:49:47.448700 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0920 05:49:47.448841 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0920 05:49:47.448970 9338 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0920 05:49:47.449023 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0920 05:49:47.449166 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0920 05:49:47.449344 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0920 05:49:47.450650 9338 executor.go:111] Tasks: 90 done / 94 total; 2 can run I0920 05:49:47.450889 9338 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0920 05:49:47.451003 9338 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0920 05:49:47.451050 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:47.451085 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:47.451137 9338 dns.go:238] Pre-creating DNS records I0920 05:49:47.451167 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:47.457167 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:47.460448 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.460556 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.498534 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:47.601493 9338 iamrole.go:89] ListRoles: { } I0920 05:49:47.603243 9338 executor.go:111] Tasks: 0 done / 94 total; 50 can run I0920 05:49:47.603547 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:47.603672 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:47.603666 9338 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0920 05:49:47.603858 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } W0920 05:49:47.603928 9338 api.go:53] PageSize not implemented W0920 05:49:47.604047 9338 api.go:53] PageSize not implemented I0920 05:49:47.604017 9338 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0920 05:49:47.603819 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0920 05:49:47.604115 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:47.604151 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } W0920 05:49:47.604168 9338 api.go:53] PageSize not implemented I0920 05:49:47.604261 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0920 05:49:47.604325 9338 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0920 05:49:47.604422 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0920 05:49:47.604471 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0920 05:49:47.604503 9338 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0920 05:49:47.604584 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0920 05:49:47.604616 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0920 05:49:47.604622 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0920 05:49:47.604676 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:47.604711 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0920 05:49:47.606081 9338 executor.go:111] Tasks: 50 done / 94 total; 19 can run I0920 05:49:47.606203 9338 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I0920 05:49:47.606367 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I0920 05:49:47.606471 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0920 05:49:47.606681 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0920 05:49:47.606895 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:47.606975 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0920 05:49:47.607184 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:47.607231 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0920 05:49:47.607393 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:47.607455 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:47.607549 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:47.607631 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0920 05:49:47.614660 9338 executor.go:111] Tasks: 69 done / 94 total; 21 can run I0920 05:49:47.615637 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:47.615784 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:47.616147 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.616274 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:47.616552 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.616769 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.617030 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.619023 9338 executor.go:111] Tasks: 90 done / 94 total; 2 can run I0920 05:49:47.619284 9338 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0920 05:49:47.619405 9338 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0920 05:49:47.623135 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0920 05:49:47.623196 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0920 05:49:47.623232 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0920 05:49:47.623264 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0920 05:49:47.623299 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0920 05:49:47.623381 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0920 05:49:47.623418 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0920 05:49:47.623451 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0920 05:49:47.623485 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0920 05:49:47.623521 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0920 05:49:47.623556 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0920 05:49:47.623598 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0920 05:49:47.623640 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0920 05:49:47.623673 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0920 05:49:47.623704 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0920 05:49:47.623739 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0920 05:49:47.623771 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0920 05:49:47.623805 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0920 05:49:47.623842 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:47.623879 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0920 05:49:47.623911 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0920 05:49:47.623943 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0920 05:49:47.623977 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0920 05:49:47.624015 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0920 05:49:47.624063 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0920 05:49:47.624124 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0920 05:49:47.625953 9338 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0920 05:49:47.626003 9338 keypairs.go:95] DescribeKeyPairs: { } I0920 05:49:47.626089 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0920 05:49:47.626209 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0920 05:49:47.626287 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0920 05:49:47.626348 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0920 05:49:47.626395 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0920 05:49:47.626450 9338 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0920 05:49:47.626492 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0920 05:49:47.626532 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0920 05:49:47.626578 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0920 05:49:47.626628 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0920 05:49:47.626698 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } W0920 05:49:47.626752 9338 api.go:53] PageSize not implemented I0920 05:49:47.626773 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0920 05:49:47.626799 9338 loadbalancers.go:34] PageSize not implemented I0920 05:49:47.626818 9338 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0920 05:49:47.626841 9338 targetgroups.go:35] PageSize not implemented I0920 05:49:47.626857 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0920 05:49:47.626904 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:47.626931 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:47.626979 9338 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0920 05:49:47.627015 9338 iamrole.go:89] ListRoles: { } I0920 05:49:47.627056 9338 internetgateways.go:101] DescribeInternetGateways: { } I0920 05:49:47.627100 9338 routetable.go:53] DescribeRouteTables: { } I0920 05:49:47.627136 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:47.627573 9338 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } I0920 05:49:47.627575 9338 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } volume:vol-2 ok I0920 05:49:47.627680 9338 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } autoscaling-group:master-us-test-1a.masters.externallb.example.com ok iam-instance-profile:masters.externallb.example.com ok I0920 05:49:47.627686 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-group:nodes.externallb.example.com ok I0920 05:49:47.627769 9338 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0920 05:49:47.627787 9338 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0920 05:49:47.627672 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.externallb.example.com. ok I0920 05:49:47.627871 9338 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-2 ok I0920 05:49:47.627893 9338 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } autoscaling-config:lt-1 ok iam-instance-profile:nodes.externallb.example.com ok I0920 05:49:47.627998 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0920 05:49:47.628141 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:47.628164 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0920 05:49:47.628208 9338 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0920 05:49:47.628270 9338 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0920 05:49:47.628371 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:47.628456 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0920 05:49:47.628809 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:47.628832 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0920 05:49:47.628907 9338 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.externallb.example.com" } I0920 05:49:47.628967 9338 iamrole.go:180] ListAttachedRolePolicies: nodes.externallb.example.com I0920 05:49:47.628982 9338 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0920 05:49:47.628947 9338 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } I0920 05:49:47.629016 9338 iamrole.go:137] DeleteRole: { RoleName: "nodes.externallb.example.com" } subnet:subnet-1 ok iam-role:nodes.externallb.example.com ok I0920 05:49:47.629055 9338 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.externallb.example.com" } I0920 05:49:47.629118 9338 iamrole.go:180] ListAttachedRolePolicies: masters.externallb.example.com I0920 05:49:47.629136 9338 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0920 05:49:47.629173 9338 iamrole.go:137] DeleteRole: { RoleName: "masters.externallb.example.com" } iam-role:masters.externallb.example.com ok I0920 05:49:47.629253 9338 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0920 05:49:47.629329 9338 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0920 05:49:47.629399 9338 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0920 05:49:47.629695 9338 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleExternalLB (0.36s) === RUN TestLifecycleSharedSubnet I0920 05:49:47.630049 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:47.630302 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:47.630354 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:47.630397 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:47.630446 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:47.630491 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:47.630524 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:47.630562 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:47.630590 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:47.630651 9338 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedsubnet.example.com I0920 05:49:47.655009 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:47.658265 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.658360 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.696354 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:47.800973 9338 iamrole.go:89] ListRoles: { } I0920 05:49:47.802480 9338 executor.go:111] Tasks: 0 done / 80 total; 42 can run I0920 05:49:47.803277 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } I0920 05:49:47.803302 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0920 05:49:47.803401 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:47.803423 9338 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0920 05:49:47.803649 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } I0920 05:49:47.803738 9338 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0920 05:49:47.803574 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0920 05:49:47.803944 9338 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0920 05:49:47.804119 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0920 05:49:47.804176 9338 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0920 05:49:47.804309 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0920 05:49:47.804354 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0920 05:49:47.804436 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0920 05:49:47.804465 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0920 05:49:47.804533 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:47.804623 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:47.804796 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0920 05:49:47.805241 9338 executor.go:111] Tasks: 42 done / 80 total; 16 can run I0920 05:49:47.805353 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0920 05:49:47.805475 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0920 05:49:47.805540 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0920 05:49:47.805625 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0920 05:49:47.805756 9338 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0920 05:49:47.805874 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0920 05:49:47.805870 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0920 05:49:47.805966 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0920 05:49:47.805939 9338 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] } I0920 05:49:47.806042 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0920 05:49:47.806197 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0920 05:49:47.806251 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0920 05:49:47.806385 9338 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0920 05:49:47.806472 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.sharedsubnet.example.com" }] }], VpcId: "vpc-12345678" } I0920 05:49:47.806581 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0920 05:49:47.806636 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:47.806701 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:47.807865 9338 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0920 05:49:47.810294 9338 executor.go:111] Tasks: 58 done / 80 total; 18 can run I0920 05:49:47.810510 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-2382to4000-masters.sharedsubnet.example.com" }] }] } I0920 05:49:47.810697 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.810820 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.810980 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0920 05:49:47.811232 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" }] }] } I0920 05:49:47.811454 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0920 05:49:47.811625 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-1to2379-masters.sharedsubnet.example.com" }] }] } I0920 05:49:47.811923 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0920 05:49:47.812193 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0920 05:49:47.812432 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-::/0" }] }] } I0920 05:49:47.812597 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0920 05:49:47.812742 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0920 05:49:47.812941 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-4003to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0920 05:49:47.813086 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0920 05:49:47.813231 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-udp-1to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0920 05:49:47.813397 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0920 05:49:47.813543 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.813639 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.815851 9338 executor.go:111] Tasks: 76 done / 80 total; 2 can run I0920 05:49:47.816186 9338 executor.go:111] Tasks: 78 done / 80 total; 2 can run I0920 05:49:47.816358 9338 executor.go:111] Tasks: 80 done / 80 total; 0 can run I0920 05:49:47.816415 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:47.816457 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:47.816513 9338 dns.go:238] Pre-creating DNS records I0920 05:49:47.816543 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:47.822661 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:47.825914 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.826010 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.863804 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:47.962680 9338 iamrole.go:89] ListRoles: { } I0920 05:49:47.964206 9338 executor.go:111] Tasks: 0 done / 80 total; 42 can run I0920 05:49:47.964636 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0920 05:49:47.964735 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:47.964810 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0920 05:49:47.964920 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0920 05:49:47.964963 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0920 05:49:47.965018 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0920 05:49:47.965202 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:47.965265 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:47.967325 9338 executor.go:111] Tasks: 42 done / 80 total; 16 can run I0920 05:49:47.967597 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0920 05:49:47.967692 9338 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0920 05:49:47.967779 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:47.967877 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0920 05:49:47.967836 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0920 05:49:47.967989 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0920 05:49:47.968174 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:47.968246 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0920 05:49:47.973558 9338 executor.go:111] Tasks: 58 done / 80 total; 18 can run I0920 05:49:47.973893 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.974071 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.974263 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.974460 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:47.976670 9338 executor.go:111] Tasks: 76 done / 80 total; 2 can run I0920 05:49:47.976931 9338 executor.go:111] Tasks: 78 done / 80 total; 2 can run I0920 05:49:47.977070 9338 executor.go:111] Tasks: 80 done / 80 total; 0 can run I0920 05:49:47.979901 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0920 05:49:47.979963 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0920 05:49:47.980030 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0920 05:49:47.980065 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0920 05:49:47.980097 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0920 05:49:47.980129 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0920 05:49:47.980160 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0920 05:49:47.980192 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0920 05:49:47.980228 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0920 05:49:47.980265 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0920 05:49:47.980306 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0920 05:49:47.980339 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0920 05:49:47.980370 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0920 05:49:47.980401 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0920 05:49:47.980435 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0920 05:49:47.980469 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0920 05:49:47.980506 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0920 05:49:47.980538 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0920 05:49:47.980572 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0920 05:49:47.980607 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0920 05:49:47.980645 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0920 05:49:47.982050 9338 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0920 05:49:47.982080 9338 keypairs.go:95] DescribeKeyPairs: { } I0920 05:49:47.982122 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0920 05:49:47.982176 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0920 05:49:47.982232 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0920 05:49:47.982290 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0920 05:49:47.982331 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0920 05:49:47.982371 9338 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0920 05:49:47.982411 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0920 05:49:47.982445 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0920 05:49:47.982482 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0920 05:49:47.982525 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0920 05:49:47.982560 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } W0920 05:49:47.982608 9338 api.go:53] PageSize not implemented I0920 05:49:47.982629 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0920 05:49:47.982656 9338 loadbalancers.go:34] PageSize not implemented I0920 05:49:47.982676 9338 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0920 05:49:47.982701 9338 targetgroups.go:35] PageSize not implemented I0920 05:49:47.982720 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0920 05:49:47.982768 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:47.982797 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:47.982845 9338 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0920 05:49:47.982881 9338 iamrole.go:89] ListRoles: { } I0920 05:49:47.982928 9338 internetgateways.go:101] DescribeInternetGateways: { } I0920 05:49:47.982965 9338 routetable.go:53] DescribeRouteTables: { } autoscaling-config:lt-2 ok I0920 05:49:47.983348 9338 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0920 05:49:47.983387 9338 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } autoscaling-group:master-us-test-1a.masters.sharedsubnet.example.com ok I0920 05:49:47.983435 9338 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } iam-instance-profile:nodes.sharedsubnet.example.com ok autoscaling-group:nodes.sharedsubnet.example.com ok keypair:key-1 ok I0920 05:49:47.983468 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:47.983348 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.sharedsubnet.example.com. ok I0920 05:49:47.983525 9338 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0920 05:49:47.983610 9338 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } iam-instance-profile:masters.sharedsubnet.example.com ok I0920 05:49:47.983530 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } W0920 05:49:47.983745 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:47.983770 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0920 05:49:47.983808 9338 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0920 05:49:47.983866 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:47.983918 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0920 05:49:47.984031 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:47.984044 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0920 05:49:47.984076 9338 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-1 ok I0920 05:49:47.984182 9338 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedsubnet.example.com" } I0920 05:49:47.984235 9338 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0920 05:49:47.984250 9338 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0920 05:49:47.984293 9338 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedsubnet.example.com" } iam-role:nodes.sharedsubnet.example.com ok I0920 05:49:47.984329 9338 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedsubnet.example.com" } I0920 05:49:47.984419 9338 iamrole.go:180] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0920 05:49:47.984435 9338 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0920 05:49:47.984470 9338 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedsubnet.example.com" } iam-role:masters.sharedsubnet.example.com ok I0920 05:49:47.984731 9338 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedSubnet (0.35s) === RUN TestLifecyclePrivateSharedSubnet I0920 05:49:47.985000 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:47.985246 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:47.985292 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:47.985337 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:47.985384 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:47.985423 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:47.985451 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:47.985484 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:47.985509 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:47.985572 9338 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-subnet.example.com I0920 05:49:48.015548 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:48.019527 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.019634 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.019707 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.059516 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:48.161799 9338 iamrole.go:89] ListRoles: { } I0920 05:49:48.164313 9338 executor.go:111] Tasks: 0 done / 107 total; 45 can run I0920 05:49:48.164642 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } I0920 05:49:48.164651 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0920 05:49:48.164780 9338 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.private-shared-subnet.example.com" }] } I0920 05:49:48.164911 9338 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0920 05:49:48.164961 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:48.165080 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0920 05:49:48.165076 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } I0920 05:49:48.167091 9338 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0920 05:49:48.167866 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } I0920 05:49:48.165137 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0920 05:49:48.168124 9338 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" }] } I0920 05:49:48.168340 9338 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-subnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0920 05:49:48.168831 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0920 05:49:48.168998 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:48.169073 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0920 05:49:48.169248 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0920 05:49:48.169307 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0920 05:49:48.169494 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:48.170756 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0920 05:49:48.172681 9338 executor.go:111] Tasks: 45 done / 107 total; 24 can run I0920 05:49:48.172807 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0920 05:49:48.172972 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I0920 05:49:48.173122 9338 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0920 05:49:48.173156 9338 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0920 05:49:48.173150 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0920 05:49:48.173239 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0920 05:49:48.173298 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0920 05:49:48.173424 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0920 05:49:48.173563 9338 tags.go:42] CreateTags { Resources: ["subnet-abcdef"], Tags: [ { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] } I0920 05:49:48.173568 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0920 05:49:48.173683 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0920 05:49:48.173666 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0920 05:49:48.173728 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0920 05:49:48.173779 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0920 05:49:48.173830 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:48.173813 9338 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0920 05:49:48.173875 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0920 05:49:48.174070 9338 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0920 05:49:48.174073 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0920 05:49:48.174330 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" }] }], VpcId: "vpc-12345678" } I0920 05:49:48.174449 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0920 05:49:48.174580 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0920 05:49:48.174687 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0920 05:49:48.174727 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "masters.private-shared-subnet.example.com" }] }], VpcId: "vpc-12345678" } I0920 05:49:48.174823 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0920 05:49:48.174940 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0920 05:49:48.174997 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0920 05:49:48.175040 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0920 05:49:48.175090 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0920 05:49:48.175124 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "api-elb.private-shared-subnet.example.com" }] }], VpcId: "vpc-12345678" } I0920 05:49:48.175218 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0920 05:49:48.175278 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0920 05:49:48.175328 9338 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] } I0920 05:49:48.175436 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0920 05:49:48.175485 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:48.175555 9338 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0920 05:49:48.179118 9338 executor.go:111] Tasks: 69 done / 107 total; 31 can run I0920 05:49:48.179343 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-ingress-tcp-22to22-bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } W0920 05:49:48.179635 9338 api.go:53] PageSize not implemented W0920 05:49:48.179683 9338 api.go:53] PageSize not implemented I0920 05:49:48.179592 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0920 05:49:48.179831 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0920 05:49:48.179839 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0920 05:49:48.179941 9338 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"], Tags: [{ Key: "Name", Value: "api.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0920 05:49:48.180049 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0920 05:49:48.180094 9338 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0920 05:49:48.180038 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0920 05:49:48.180154 9338 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0920 05:49:48.180240 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0920 05:49:48.180266 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.180307 9338 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"], Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0920 05:49:48.180363 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0920 05:49:48.180388 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.180391 9338 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0920 05:49:48.180441 9338 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0920 05:49:48.180580 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0920 05:49:48.180804 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0920 05:49:48.181009 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-2382to4000-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0920 05:49:48.181202 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0920 05:49:48.181366 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.181470 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I0920 05:49:48.181658 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0920 05:49:48.181833 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0920 05:49:48.181967 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0920 05:49:48.182099 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0920 05:49:48.182309 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0920 05:49:48.182461 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.182577 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.182658 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0920 05:49:48.182843 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0920 05:49:48.183028 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I0920 05:49:48.183152 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } I0920 05:49:48.183257 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-1to2379-masters.private-shared-subnet.example.com" }] }] } I0920 05:49:48.183432 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-subnet.example.com" }] }] } I0920 05:49:48.183555 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.183664 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0920 05:49:48.183820 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-4003to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0920 05:49:48.183962 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0920 05:49:48.184114 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-udp-1to65535-masters.private-shared-subnet.example.com" }] }] } I0920 05:49:48.185177 9338 executor.go:111] Tasks: 100 done / 107 total; 4 can run I0920 05:49:48.185274 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0920 05:49:48.185490 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0920 05:49:48.185681 9338 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0920 05:49:48.185904 9338 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0920 05:49:48.185972 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:48.186021 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:48.186078 9338 dns.go:238] Pre-creating DNS records I0920 05:49:48.186103 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:48.192665 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:48.196781 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.196878 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.196948 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.234243 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:48.337501 9338 iamrole.go:89] ListRoles: { } I0920 05:49:48.339694 9338 executor.go:111] Tasks: 0 done / 107 total; 45 can run I0920 05:49:48.339882 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0920 05:49:48.340080 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0920 05:49:48.340144 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0920 05:49:48.340235 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0920 05:49:48.340289 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:48.340422 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0920 05:49:48.340712 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:48.341923 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:48.346327 9338 executor.go:111] Tasks: 45 done / 107 total; 24 can run I0920 05:49:48.346773 9338 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0920 05:49:48.346831 9338 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0920 05:49:48.346860 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0920 05:49:48.346657 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0920 05:49:48.347025 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0920 05:49:48.347890 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0920 05:49:48.348091 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0920 05:49:48.348158 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0920 05:49:48.348278 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0920 05:49:48.348428 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:48.348478 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0920 05:49:48.348528 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0920 05:49:48.348635 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0920 05:49:48.348771 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0920 05:49:48.348821 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I0920 05:49:48.348955 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:48.352749 9338 executor.go:111] Tasks: 69 done / 107 total; 31 can run I0920 05:49:48.352990 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } W0920 05:49:48.353078 9338 api.go:53] PageSize not implemented I0920 05:49:48.353116 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I0920 05:49:48.353225 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0920 05:49:48.353175 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.353280 9338 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } W0920 05:49:48.353344 9338 api.go:53] PageSize not implemented I0920 05:49:48.353393 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.353381 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I0920 05:49:48.353492 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0920 05:49:48.353535 9338 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0920 05:49:48.353520 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.354057 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.354212 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.355660 9338 executor.go:111] Tasks: 100 done / 107 total; 4 can run W0920 05:49:48.355796 9338 api.go:53] PageSize not implemented I0920 05:49:48.355754 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0920 05:49:48.355842 9338 dnsname.go:121] AliasTarget for "api.private-shared-subnet.example.com" is "api-private-shared-subnet-n2f8ak.elb.cloudmock.com" I0920 05:49:48.355827 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I0920 05:49:48.356013 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } W0920 05:49:48.356072 9338 api.go:53] PageSize not implemented I0920 05:49:48.356090 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0920 05:49:48.356287 9338 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0920 05:49:48.356524 9338 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0920 05:49:48.360509 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0920 05:49:48.360575 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0920 05:49:48.360613 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0920 05:49:48.360647 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0920 05:49:48.360682 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0920 05:49:48.360717 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0920 05:49:48.360749 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0920 05:49:48.360783 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0920 05:49:48.360819 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0920 05:49:48.360859 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0920 05:49:48.360895 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0920 05:49:48.360940 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0920 05:49:48.360974 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0920 05:49:48.361007 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0920 05:49:48.361037 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0920 05:49:48.361073 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0920 05:49:48.361111 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0920 05:49:48.361149 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0920 05:49:48.361185 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0920 05:49:48.361220 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0920 05:49:48.361257 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0920 05:49:48.361294 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0920 05:49:48.361330 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0920 05:49:48.361368 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0920 05:49:48.361405 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0920 05:49:48.363284 9338 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0920 05:49:48.363350 9338 keypairs.go:95] DescribeKeyPairs: { } I0920 05:49:48.363421 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0920 05:49:48.363490 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0920 05:49:48.363573 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0920 05:49:48.363635 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0920 05:49:48.363679 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0920 05:49:48.363720 9338 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0920 05:49:48.363763 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0920 05:49:48.363799 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0920 05:49:48.363840 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0920 05:49:48.363888 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0920 05:49:48.363927 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } W0920 05:49:48.363974 9338 api.go:53] PageSize not implemented I0920 05:49:48.363994 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I0920 05:49:48.364047 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0920 05:49:48.364075 9338 loadbalancers.go:34] PageSize not implemented I0920 05:49:48.364093 9338 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0920 05:49:48.364118 9338 targetgroups.go:35] PageSize not implemented I0920 05:49:48.364136 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0920 05:49:48.364189 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:48.364217 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:48.364264 9338 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0920 05:49:48.364304 9338 iamrole.go:89] ListRoles: { } I0920 05:49:48.364350 9338 internetgateways.go:101] DescribeInternetGateways: { } I0920 05:49:48.364396 9338 routetable.go:53] DescribeRouteTables: { } autoscaling-group:nodes.private-shared-subnet.example.com ok I0920 05:49:48.364912 9338 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0920 05:49:48.364982 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } autoscaling-group:bastion.private-shared-subnet.example.com ok I0920 05:49:48.365049 9338 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0920 05:49:48.365040 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } I0920 05:49:48.365145 9338 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } iam-instance-profile:masters.private-shared-subnet.example.com ok I0920 05:49:48.364954 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } W0920 05:49:48.365163 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions route53-record:Z1AFAKE1ZON3YO/A/api.private-shared-subnet.example.com ok I0920 05:49:48.365145 9338 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0920 05:49:48.365180 9338 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } load-balancer:bastion-private-shared-su-5ol32q ok I0920 05:49:48.365205 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0920 05:49:48.365237 9338 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } iam-instance-profile:bastions.private-shared-subnet.example.com ok I0920 05:49:48.365236 9338 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0920 05:49:48.365260 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } load-balancer:api-private-shared-subnet-n2f8ak ok I0920 05:49:48.365283 9338 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } autoscaling-group:master-us-test-1a.masters.private-shared-subnet.example.com ok I0920 05:49:48.365330 9338 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } iam-instance-profile:nodes.private-shared-subnet.example.com ok I0920 05:49:48.365324 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] } ] } W0920 05:49:48.365542 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:48.365564 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok autoscaling-config:lt-2 ok I0920 05:49:48.365615 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:48.365670 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0920 05:49:48.365749 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:48.365777 9338 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-1 ok autoscaling-config:lt-3 ok I0920 05:49:48.365909 9338 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0920 05:49:48.365974 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0920 05:49:48.366050 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:48.366080 9338 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-subnet.example.com" } I0920 05:49:48.366170 9338 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0920 05:49:48.366110 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0920 05:49:48.366200 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:48.366190 9338 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0920 05:49:48.366214 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0920 05:49:48.366226 9338 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-subnet.example.com" } iam-role:masters.private-shared-subnet.example.com ok I0920 05:49:48.366245 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0920 05:49:48.366263 9338 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-subnet.example.com" } I0920 05:49:48.366315 9338 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0920 05:49:48.366331 9338 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0920 05:49:48.366361 9338 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-subnet.example.com" } iam-role:nodes.private-shared-subnet.example.com ok I0920 05:49:48.366293 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W0920 05:49:48.366394 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:48.366393 9338 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-subnet.example.com" } I0920 05:49:48.366436 9338 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0920 05:49:48.366448 9338 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0920 05:49:48.366476 9338 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-subnet.example.com" } iam-role:bastions.private-shared-subnet.example.com ok I0920 05:49:48.366411 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0920 05:49:48.366833 9338 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedSubnet (0.38s) === RUN TestLifecyclePrivateSharedIP I0920 05:49:48.367168 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:48.367421 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:48.367472 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:48.367520 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:48.367572 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:48.367617 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:48.367646 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:48.367683 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:48.367707 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:48.367773 9338 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-ip.example.com I0920 05:49:48.396013 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:48.399700 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.399791 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.399856 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.437839 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:48.543734 9338 iamrole.go:89] ListRoles: { } I0920 05:49:48.545919 9338 executor.go:111] Tasks: 0 done / 116 total; 45 can run I0920 05:49:48.546227 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:48.546266 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } I0920 05:49:48.546324 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:48.546380 9338 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0920 05:49:48.546347 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:48.546583 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "private-shared-ip.example.com" }] }] } I0920 05:49:48.546879 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0920 05:49:48.546941 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } I0920 05:49:48.547072 9338 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0920 05:49:48.547238 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } I0920 05:49:48.547207 9338 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0920 05:49:48.547362 9338 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0920 05:49:48.547447 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0920 05:49:48.547523 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0920 05:49:48.547680 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0920 05:49:48.547964 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0920 05:49:48.548048 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0920 05:49:48.548142 9338 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0920 05:49:48.548326 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0920 05:49:48.548942 9338 executor.go:111] Tasks: 45 done / 116 total; 26 can run I0920 05:49:48.549077 9338 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0920 05:49:48.549095 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I0920 05:49:48.549194 9338 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0920 05:49:48.549282 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0920 05:49:48.549397 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0920 05:49:48.549422 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0920 05:49:48.549452 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0920 05:49:48.549565 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0920 05:49:48.549617 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0920 05:49:48.549672 9338 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0920 05:49:48.549851 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" } ] }], VpcId: "vpc-12345678" } I0920 05:49:48.549913 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0920 05:49:48.550135 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0920 05:49:48.550231 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0920 05:49:48.550373 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0920 05:49:48.550430 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:48.550467 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0920 05:49:48.550533 9338 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0920 05:49:48.550586 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0920 05:49:48.550705 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] }] } I0920 05:49:48.550805 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0920 05:49:48.550911 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] }] } I0920 05:49:48.550893 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" } ] }], VpcId: "vpc-12345678" } I0920 05:49:48.550999 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0920 05:49:48.551095 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0920 05:49:48.551182 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0920 05:49:48.551229 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0920 05:49:48.551294 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0920 05:49:48.551480 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0920 05:49:48.551591 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0920 05:49:48.551526 9338 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-12345678" } I0920 05:49:48.551646 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0920 05:49:48.551705 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0920 05:49:48.551798 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0920 05:49:48.551873 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0920 05:49:48.552042 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0920 05:49:48.552070 9338 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "private-shared-ip.example.com" } ] }], VpcId: "vpc-12345678" } I0920 05:49:48.552153 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0920 05:49:48.552224 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0920 05:49:48.552292 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:48.552364 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0920 05:49:48.552430 9338 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] } I0920 05:49:48.552576 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0920 05:49:48.552018 9338 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0920 05:49:48.552663 9338 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0920 05:49:48.552806 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0920 05:49:48.552878 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:48.552943 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:48.555862 9338 executor.go:111] Tasks: 71 done / 116 total; 36 can run W0920 05:49:48.556113 9338 api.go:53] PageSize not implemented I0920 05:49:48.556142 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:48.556415 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0920 05:49:48.556411 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0920 05:49:48.556513 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0920 05:49:48.556562 9338 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"], Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "api.private-shared-ip.example.com" }] } I0920 05:49:48.556756 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0920 05:49:48.556585 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0920 05:49:48.556816 9338 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0920 05:49:48.556936 9338 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } W0920 05:49:48.557070 9338 api.go:53] PageSize not implemented I0920 05:49:48.556967 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:48.557153 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0920 05:49:48.557362 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I0920 05:49:48.558838 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0920 05:49:48.558917 9338 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"], Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0920 05:49:48.558992 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0920 05:49:48.558702 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0920 05:49:48.559028 9338 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0920 05:49:48.559084 9338 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0920 05:49:48.559098 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0920 05:49:48.559476 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-4003to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0920 05:49:48.560560 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0920 05:49:48.560788 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:48.560909 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:48.561045 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0920 05:49:48.561243 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-2382to4000-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0920 05:49:48.561408 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0920 05:49:48.561565 9338 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0920 05:49:48.561654 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-ingress-tcp-22to22-bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0920 05:49:48.561818 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-ip.example.com" }] }] } I0920 05:49:48.561944 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0920 05:49:48.562039 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0920 05:49:48.562081 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } I0920 05:49:48.562174 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0920 05:49:48.562316 9338 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-2" } I0920 05:49:48.562369 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.562460 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0920 05:49:48.562590 9338 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-2" } I0920 05:49:48.562636 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-udp-1to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0920 05:49:48.562775 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-1to2379-masters.private-shared-ip.example.com" }] }] } I0920 05:49:48.562907 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0920 05:49:48.563040 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0920 05:49:48.563140 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0920 05:49:48.563272 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0920 05:49:48.563340 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0920 05:49:48.563486 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.563569 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0920 05:49:48.563690 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.563779 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0920 05:49:48.563901 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-masters.private-shared-ip.example.com" }] }] } I0920 05:49:48.564028 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0920 05:49:48.564063 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.564111 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0920 05:49:48.564156 9338 tags.go:42] CreateTags { Resources: ["eipalloc-12345678"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "shared" }] } I0920 05:49:48.564247 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.564383 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.566609 9338 executor.go:111] Tasks: 107 done / 116 total; 5 can run I0920 05:49:48.566717 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:48.566806 9338 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-2"] }] } I0920 05:49:48.566901 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0920 05:49:48.566930 9338 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-12345678", SubnetId: "subnet-2", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0920 05:49:48.567024 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0920 05:49:48.567068 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0920 05:49:48.567108 9338 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0920 05:49:48.567077 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0920 05:49:48.567207 9338 executor.go:111] Tasks: 112 done / 116 total; 4 can run I0920 05:49:48.567275 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:48.567462 9338 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I0920 05:49:48.567576 9338 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0920 05:49:48.567642 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:48.567691 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:48.567773 9338 dns.go:238] Pre-creating DNS records I0920 05:49:48.567816 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:48.575605 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0920 05:49:48.579206 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.579297 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.579366 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.617011 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:48.714071 9338 iamrole.go:89] ListRoles: { } I0920 05:49:48.715778 9338 executor.go:111] Tasks: 0 done / 116 total; 45 can run I0920 05:49:48.718486 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] }] } I0920 05:49:48.718989 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0920 05:49:48.718803 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:48.719082 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0920 05:49:48.719201 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0920 05:49:48.719333 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:48.719532 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:48.720892 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0920 05:49:48.723288 9338 executor.go:111] Tasks: 45 done / 116 total; 26 can run I0920 05:49:48.723443 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] }] } I0920 05:49:48.723561 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0920 05:49:48.723631 9338 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0920 05:49:48.723685 9338 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0920 05:49:48.723594 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0920 05:49:48.723811 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0920 05:49:48.723878 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:48.723972 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0920 05:49:48.724233 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:48.724299 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] }] } I0920 05:49:48.724457 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0920 05:49:48.724568 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0920 05:49:48.724718 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0920 05:49:48.724764 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I0920 05:49:48.724895 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0920 05:49:48.725016 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0920 05:49:48.725144 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0920 05:49:48.725261 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:48.725317 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0920 05:49:48.725366 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0920 05:49:48.729071 9338 executor.go:111] Tasks: 71 done / 116 total; 36 can run I0920 05:49:48.729211 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0920 05:49:48.729392 9338 api.go:53] PageSize not implemented I0920 05:49:48.729318 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.729432 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0920 05:49:48.729535 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0920 05:49:48.729498 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0920 05:49:48.729575 9338 api.go:53] PageSize not implemented I0920 05:49:48.729633 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.729615 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0920 05:49:48.729754 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0920 05:49:48.729799 9338 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0920 05:49:48.729894 9338 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0920 05:49:48.729919 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.730107 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.730333 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:48.730718 9338 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0920 05:49:48.730834 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0920 05:49:48.731118 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0920 05:49:48.731340 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.731581 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0920 05:49:48.732615 9338 executor.go:111] Tasks: 107 done / 116 total; 5 can run I0920 05:49:48.732715 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0920 05:49:48.732805 9338 dnsname.go:121] AliasTarget for "api.private-shared-ip.example.com" is "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com" W0920 05:49:48.732769 9338 api.go:53] PageSize not implemented I0920 05:49:48.732841 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0920 05:49:48.732840 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:48.732987 9338 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } W0920 05:49:48.733024 9338 api.go:53] PageSize not implemented I0920 05:49:48.733046 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0920 05:49:48.733203 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0920 05:49:48.733342 9338 executor.go:111] Tasks: 112 done / 116 total; 4 can run I0920 05:49:48.733465 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:48.733573 9338 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0920 05:49:48.737290 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0920 05:49:48.737357 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0920 05:49:48.737395 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0920 05:49:48.737429 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0920 05:49:48.737466 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0920 05:49:48.737504 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0920 05:49:48.737537 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0920 05:49:48.737572 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0920 05:49:48.737610 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0920 05:49:48.737651 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0920 05:49:48.737692 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0920 05:49:48.737736 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0920 05:49:48.737773 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0920 05:49:48.737807 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0920 05:49:48.737844 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0920 05:49:48.737881 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0920 05:49:48.737915 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0920 05:49:48.737952 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0920 05:49:48.737991 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0920 05:49:48.738029 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0920 05:49:48.738067 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0920 05:49:48.738105 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0920 05:49:48.738143 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:48.738183 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0920 05:49:48.738217 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0920 05:49:48.738255 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0920 05:49:48.738294 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0920 05:49:48.738329 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0920 05:49:48.738369 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0920 05:49:48.738407 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0920 05:49:48.740025 9338 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0920 05:49:48.740055 9338 keypairs.go:95] DescribeKeyPairs: { } I0920 05:49:48.740098 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0920 05:49:48.740165 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0920 05:49:48.740244 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0920 05:49:48.740305 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0920 05:49:48.740349 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0920 05:49:48.740392 9338 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0920 05:49:48.740431 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0920 05:49:48.740475 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0920 05:49:48.740530 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0920 05:49:48.740586 9338 routetable.go:53] DescribeRouteTables: { } I0920 05:49:48.740615 9338 natgateway.go:117] DescribeNatGateways: { } I0920 05:49:48.740663 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0920 05:49:48.740701 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } W0920 05:49:48.740748 9338 api.go:53] PageSize not implemented I0920 05:49:48.740766 9338 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I0920 05:49:48.740820 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0920 05:49:48.740850 9338 loadbalancers.go:34] PageSize not implemented I0920 05:49:48.740868 9338 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0920 05:49:48.740894 9338 targetgroups.go:35] PageSize not implemented I0920 05:49:48.740912 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0920 05:49:48.740969 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:48.740998 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:48.741040 9338 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0920 05:49:48.741078 9338 iamrole.go:89] ListRoles: { } I0920 05:49:48.741132 9338 internetgateways.go:101] DescribeInternetGateways: { } I0920 05:49:48.741179 9338 routetable.go:53] DescribeRouteTables: { } I0920 05:49:48.741215 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I0920 05:49:48.741254 9338 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0920 05:49:48.741289 9338 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0920 05:49:48.741861 9338 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0920 05:49:48.741929 9338 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } I0920 05:49:48.741907 9338 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } iam-instance-profile:nodes.private-shared-ip.example.com ok volume:vol-2 ok I0920 05:49:48.741960 9338 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } autoscaling-group:nodes.private-shared-ip.example.com ok I0920 05:49:48.742004 9338 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } autoscaling-group:master-us-test-1a.masters.private-shared-ip.example.com ok iam-instance-profile:bastions.private-shared-ip.example.com ok I0920 05:49:48.742016 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:48.742039 9338 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0920 05:49:48.742098 9338 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } autoscaling-group:bastion.private-shared-ip.example.com ok I0920 05:49:48.742101 9338 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } iam-instance-profile:masters.private-shared-ip.example.com ok load-balancer:bastion-private-shared-ip-eepmph ok I0920 05:49:48.742056 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:48.742237 9338 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.private-shared-ip.example.com. ok I0920 05:49:48.742099 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } load-balancer:api-private-shared-ip-exa-ohatqj ok W0920 05:49:48.742335 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:48.742354 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0920 05:49:48.742398 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0920 05:49:48.742462 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } W0920 05:49:48.742578 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:48.742594 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0920 05:49:48.742641 9338 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0920 05:49:48.742704 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0920 05:49:48.742761 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0920 05:49:48.742841 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:48.742856 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok autoscaling-config:lt-2 ok autoscaling-config:lt-3 ok autoscaling-config:lt-1 ok I0920 05:49:48.742929 9338 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0920 05:49:48.742999 9338 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0920 05:49:48.743155 9338 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-ip.example.com" } I0920 05:49:48.743215 9338 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0920 05:49:48.743176 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:48.743231 9338 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0920 05:49:48.743269 9338 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-ip.example.com" } iam-role:nodes.private-shared-ip.example.com ok I0920 05:49:48.743301 9338 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-ip.example.com" } I0920 05:49:48.743368 9338 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0920 05:49:48.743275 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0920 05:49:48.743394 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:48.743416 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0920 05:49:48.743380 9338 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0920 05:49:48.743454 9338 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0920 05:49:48.743507 9338 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0920 05:49:48.743503 9338 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-ip.example.com" } iam-role:bastions.private-shared-ip.example.com ok I0920 05:49:48.743555 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0920 05:49:48.743569 9338 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-ip.example.com" } I0920 05:49:48.743648 9338 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0920 05:49:48.743664 9338 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0920 05:49:48.743603 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } I0920 05:49:48.743693 9338 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-ip.example.com" } iam-role:masters.private-shared-ip.example.com ok W0920 05:49:48.743702 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:48.743731 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0920 05:49:48.743795 9338 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0920 05:49:48.743853 9338 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0920 05:49:48.744137 9338 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedIP (0.38s) === RUN TestLifecycleNodeTerminationHandlerQueueProcessor I0920 05:49:48.744471 9338 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:48.744716 9338 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:48.744761 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:48.744804 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:48.744856 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:48.744901 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:48.744930 9338 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:48.744965 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:48.744993 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:48.745059 9338 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster nthsqsresources.longclustername.example.com I0920 05:49:48.770829 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0920 05:49:48.776581 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:48.776724 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:49.181136 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:49.308385 9338 iamrole.go:89] ListRoles: { } I0920 05:49:49.310106 9338 executor.go:111] Tasks: 0 done / 101 total; 45 can run I0920 05:49:49.310507 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0920 05:49:49.310772 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:49.310773 9338 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0920 05:49:49.310868 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:49.310997 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:49.311054 9338 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0920 05:49:49.311105 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } I0920 05:49:49.311276 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0920 05:49:49.311272 9338 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I0920 05:49:49.311528 9338 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } I0920 05:49:49.311606 9338 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I0920 05:49:49.312698 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0920 05:49:49.312999 9338 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0920 05:49:49.313097 9338 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0920 05:49:49.313173 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0920 05:49:49.313423 9338 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.nthsqsresources.longclustername.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0920 05:49:49.313718 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0920 05:49:49.313809 9338 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0920 05:49:49.314039 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0920 05:49:49.314147 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:49.314232 9338 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.nthsqsresources.longclustername.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0920 05:49:49.314489 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0920 05:49:49.314582 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0920 05:49:49.316805 9338 executor.go:111] Tasks: 45 done / 101 total; 24 can run I0920 05:49:49.316985 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I0920 05:49:49.317133 9338 iamrole.go:161] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0920 05:49:49.317844 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0920 05:49:49.318044 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0920 05:49:49.318142 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0920 05:49:49.318277 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0920 05:49:49.318230 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:49.318371 9338 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0920 05:49:49.318413 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0920 05:49:49.318427 9338 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" } ] }], VpcId: "vpc-1" } I0920 05:49:49.318575 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:49.318838 9338 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0920 05:49:49.318932 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0920 05:49:49.319105 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0920 05:49:49.319051 9338 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0920 05:49:49.319198 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:49.319261 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I0920 05:49:49.319413 9338 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0920 05:49:49.319531 9338 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0920 05:49:49.319597 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0920 05:49:49.319739 9338 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0920 05:49:49.319801 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:49.319919 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:49.320036 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0920 05:49:49.320112 9338 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "nthsqsresources.longclustername.example.com" } ] } I0920 05:49:49.320237 9338 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0920 05:49:49.320361 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0920 05:49:49.320802 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:49.320899 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0920 05:49:49.321407 9338 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\",\n \"sqs:DeleteMessage\",\n \"sqs:ReceiveMessage\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:CompleteLifecycleAction\",\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0920 05:49:49.323226 9338 executor.go:111] Tasks: 69 done / 101 total; 26 can run I0920 05:49:49.323559 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0920 05:49:49.323827 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-2382to4000-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0920 05:49:49.324208 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-udp-1to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0920 05:49:49.324504 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0920 05:49:49.324675 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:49.324818 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0920 05:49:49.324859 9338 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0920 05:49:49.324897 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:49.324979 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-4003to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0920 05:49:49.325159 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:49.325288 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0920 05:49:49.325524 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0920 05:49:49.325710 9338 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0920 05:49:49.325782 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" }] }] } I0920 05:49:49.325937 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0920 05:49:49.326101 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-1to2379-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0920 05:49:49.326263 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:49.326499 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:49.326617 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0920 05:49:49.327009 9338 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0920 05:49:49.327118 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" }] }] } I0920 05:49:49.327385 9338 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0920 05:49:49.327552 9338 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.nthsqsresources.longclustername.example.com" }] }] } I0920 05:49:49.327743 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:49.327863 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:49.329419 9338 executor.go:111] Tasks: 95 done / 101 total; 2 can run I0920 05:49:49.329935 9338 executor.go:111] Tasks: 97 done / 101 total; 4 can run I0920 05:49:49.330423 9338 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0920 05:49:49.330479 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:49.330520 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:49.330589 9338 dns.go:238] Pre-creating DNS records I0920 05:49:49.330629 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:49.336474 9338 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0920 05:49:49.339883 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:49.340077 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:49.377456 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:49.507023 9338 iamrole.go:89] ListRoles: { } I0920 05:49:49.508680 9338 executor.go:111] Tasks: 0 done / 101 total; 45 can run I0920 05:49:49.509151 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0920 05:49:49.509232 9338 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0920 05:49:49.509333 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0920 05:49:49.509366 9338 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0920 05:49:49.509413 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:49.509449 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:49.509507 9338 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0920 05:49:49.509584 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0920 05:49:49.511867 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0920 05:49:49.512160 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0920 05:49:49.515599 9338 executor.go:111] Tasks: 45 done / 101 total; 24 can run I0920 05:49:49.515748 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:49.516517 9338 iamrole.go:161] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0920 05:49:49.516701 9338 iamrole.go:161] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0920 05:49:49.516450 9338 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:49.516815 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0920 05:49:49.516979 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0920 05:49:49.517135 9338 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:49.517174 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0920 05:49:49.517303 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I0920 05:49:49.517478 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:49.517528 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I0920 05:49:49.517660 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0920 05:49:49.521519 9338 executor.go:111] Tasks: 69 done / 101 total; 26 can run I0920 05:49:49.521709 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:49.521887 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:49.522062 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:49.522222 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:49.522440 9338 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0920 05:49:49.522505 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:49.522643 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:49.525034 9338 executor.go:111] Tasks: 95 done / 101 total; 2 can run I0920 05:49:49.525249 9338 executor.go:111] Tasks: 97 done / 101 total; 4 can run I0920 05:49:49.525464 9338 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0920 05:49:49.528605 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0920 05:49:49.528673 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0920 05:49:49.528709 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0920 05:49:49.528740 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0920 05:49:49.528775 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0920 05:49:49.528814 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0920 05:49:49.528845 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0920 05:49:49.528879 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0920 05:49:49.528912 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0920 05:49:49.528946 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0920 05:49:49.528982 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0920 05:49:49.529019 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0920 05:49:49.529062 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0920 05:49:49.529093 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0920 05:49:49.529125 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0920 05:49:49.529161 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0920 05:49:49.529193 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0920 05:49:49.529228 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0920 05:49:49.529264 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:49.529302 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0920 05:49:49.529338 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0920 05:49:49.529371 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0920 05:49:49.529406 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0920 05:49:49.529448 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0920 05:49:49.529485 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0920 05:49:49.529522 9338 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0920 05:49:49.531018 9338 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0920 05:49:49.531056 9338 keypairs.go:95] DescribeKeyPairs: { } I0920 05:49:49.531102 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0920 05:49:49.531157 9338 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I0920 05:49:49.531216 9338 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0920 05:49:49.531278 9338 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0920 05:49:49.531326 9338 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0920 05:49:49.531386 9338 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0920 05:49:49.531428 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0920 05:49:49.531468 9338 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I0920 05:49:49.531517 9338 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0920 05:49:49.531577 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0920 05:49:49.531620 9338 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } W0920 05:49:49.531677 9338 api.go:53] PageSize not implemented I0920 05:49:49.531700 9338 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0920 05:49:49.531728 9338 loadbalancers.go:34] PageSize not implemented I0920 05:49:49.531745 9338 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0920 05:49:49.531770 9338 targetgroups.go:35] PageSize not implemented I0920 05:49:49.531789 9338 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0920 05:49:49.531842 9338 zones.go:93] ListHostedZonesPages { } I0920 05:49:49.531871 9338 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0920 05:49:49.531921 9338 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0920 05:49:49.531959 9338 iamrole.go:89] ListRoles: { } I0920 05:49:49.532009 9338 internetgateways.go:101] DescribeInternetGateways: { } I0920 05:49:49.532054 9338 routetable.go:53] DescribeRouteTables: { } I0920 05:49:49.532090 9338 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0920 05:49:49.532538 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-group:nodes.nthsqsresources.longclustername.example.com ok I0920 05:49:49.532610 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } I0920 05:49:49.532654 9338 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } W0920 05:49:49.532738 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions autoscaling-group:master-us-test-1a.masters.nthsqsresources.longclustername.example.com ok I0920 05:49:49.532756 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0920 05:49:49.532752 9338 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } iam-instance-profile:masters.nthsqsresources.longclustername.example.com ok I0920 05:49:49.532791 9338 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0920 05:49:49.532840 9338 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0920 05:49:49.532884 9338 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } I0920 05:49:49.532690 9338 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } volume:vol-2 ok autoscaling-config:lt-1 ok I0920 05:49:49.532921 9338 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.nthsqsresources.longclustername.example.com. ok iam-instance-profile:nodes.nthsqsresources.longclustername.example.com ok autoscaling-config:lt-2 ok I0920 05:49:49.533003 9338 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0920 05:49:49.533077 9338 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 } ] } W0920 05:49:49.533288 9338 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0920 05:49:49.533330 9338 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0920 05:49:49.533368 9338 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0920 05:49:49.533428 9338 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0920 05:49:49.533481 9338 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0920 05:49:49.533554 9338 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0920 05:49:49.533608 9338 iamrole.go:180] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0920 05:49:49.533578 9338 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0920 05:49:49.533623 9338 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0920 05:49:49.533692 9338 iamrole.go:137] DeleteRole: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } iam-role:nodes.nthsqsresources.longclustername.example.com ok I0920 05:49:49.533734 9338 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.nthsqsresources.longclustername.example.com" } I0920 05:49:49.533789 9338 iamrole.go:180] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0920 05:49:49.533804 9338 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0920 05:49:49.533838 9338 iamrole.go:137] DeleteRole: { RoleName: "masters.nthsqsresources.longclustername.example.com" } iam-role:masters.nthsqsresources.longclustername.example.com ok I0920 05:49:49.533904 9338 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0920 05:49:49.533980 9338 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0920 05:49:49.534052 9338 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0920 05:49:49.534330 9338 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleNodeTerminationHandlerQueueProcessor (0.79s) === RUN TestGetFilters --- PASS: TestGetFilters (0.00s) === RUN TestGetInstanceSelectorOpts --- PASS: TestGetInstanceSelectorOpts (0.00s) === RUN TestValidateAllPrivateOrPublicSubnets --- PASS: TestValidateAllPrivateOrPublicSubnets (0.00s) === RUN TestValidateUserSubnetsWithClusterSubnets --- PASS: TestValidateUserSubnetsWithClusterSubnets (0.00s) === RUN TestCreateInstanceGroup --- PASS: TestCreateInstanceGroup (0.00s) === RUN TestDecorateWithInstanceGroupSpecs --- PASS: TestDecorateWithInstanceGroupSpecs (0.00s) === RUN TestDecorateWithMixedInstancesPolicy --- PASS: TestDecorateWithMixedInstancesPolicy (0.00s) === RUN TestDecorateWithClusterAutoscalerLabels --- PASS: TestDecorateWithClusterAutoscalerLabels (0.00s) === RUN TestNewTemplateContext --- PASS: TestNewTemplateContext (0.00s) PASS ok k8s.io/kops/cmd/kops 47.259s ? k8s.io/kops/cmd/kops/util [no test files] ? k8s.io/kops/cmd/kops-controller [no test files] ? k8s.io/kops/cmd/kops-controller/controllers [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/config [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/server [no test files] === RUN TestBuildProxyRequest === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/livez === RUN TestBuildProxyRequest/http://127.0.0.1:3990/healthz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ready === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz? === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude --- PASS: TestBuildProxyRequest (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/livez (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/healthz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ready (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz? (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude (0.00s) PASS ok k8s.io/kops/cmd/kube-apiserver-healthcheck 0.011s ? k8s.io/kops/cmd/nodeup [no test files] ? k8s.io/kops/dns-controller/cmd/dns-controller [no test files] === RUN TestAliasForNodesInRole --- PASS: TestAliasForNodesInRole (0.00s) === RUN TestEnsureDotSuffix --- PASS: TestEnsureDotSuffix (0.00s) === RUN TestParseZoneSpec --- PASS: TestParseZoneSpec (0.00s) === RUN TestParseZoneRules I0920 05:48:42.479663 413 zonespec.go:75] No rules specified, will permit management of all zones --- PASS: TestParseZoneRules (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/dns 0.021s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/util 0.039s === RUN TestPodController I0920 05:48:47.704394 5148 pod.go:59] starting pod controller I0920 05:48:47.704654 5148 stoppable.go:62] shutting down controller --- PASS: TestPodController (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/watchers 0.044s === RUN TestEquivalent --- PASS: TestEquivalent (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider 0.013s Parsing flags. Getting new test interface. Running tests... === RUN TestZonesList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0000a5e00} route53_test.go:93: Got zone list: [0xc0000a5e30] route53_test.go:98: Got at least 1 zone in list:&{0xc0002b3110 0xc000012090} --- PASS: TestZonesList (0.00s) === RUN TestZonesID route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0000a5f00} route53_test.go:93: Got zone list: [0xc0000a5f30] route53_test.go:98: Got at least 1 zone in list:&{0xc0002b3110 0xc0000120a0} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess route53_test.go:79: Got zones {0xc0000a5fd0} route53_test.go:170: Successfully added managed DNS zone: &{0xc0002b32f0 0xc0000120b8} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002da130} route53_test.go:93: Got zone list: [0xc0002da160] route53_test.go:98: Got at least 1 zone in list:&{0xc0002b3110 0xc0000120d0} route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002da260} route53_test.go:93: Got zone list: [0xc0002da290] route53_test.go:98: Got at least 1 zone in list:&{0xc0002b3110 0xc0000120f0} route53_test.go:187: Successfully added resource record set: {0xc000110100 0xc000012110} --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002da510} route53_test.go:93: Got zone list: [0xc0002da540] route53_test.go:98: Got at least 1 zone in list:&{0xc0002b3110 0xc000012198} route53_test.go:199: Successfully added resource record set: {0xc000110280 0xc0000121b8} route53_test.go:118: Got 1 recordsets: [0xc0002da6e0] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002da810} route53_test.go:93: Got zone list: [0xc0002da840] route53_test.go:98: Got at least 1 zone in list:&{0xc0002b3110 0xc000012250} route53_test.go:221: Successfully added resource record set: {0xc000110400 0xc000012270} route53_test.go:228: Correctly failed to add duplicate resource record {0xc000110400 0xc000012270}: attempt to create duplicate rrset www11.example.com::A --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002daba0} route53_test.go:93: Got zone list: [0xc0002dabd0] route53_test.go:98: Got at least 1 zone in list:&{0xc0002b3110 0xc000012328} route53_test.go:246: Successfully removed resource set {0xc000110600 0xc000012348} after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002dae50} route53_test.go:93: Got zone list: [0xc0002dae80] route53_test.go:98: Got at least 1 zone in list:&{0xc0002b3110 0xc0000123d0} route53_test.go:264: Successfully removed resource set {0xc000110780 0xc0000123f0} after adding route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002db130} route53_test.go:93: Got zone list: [0xc0002db160] route53_test.go:98: Got at least 1 zone in list:&{0xc0002b3110 0xc000012480} commontests.go:192: Successfully added resource record set: {0xc000110900 0xc0000124a0} commontests.go:45: Correctly replaced resource record {0xc000110900 0xc0000124a0} -> {0xc000110a00 0xc0000124f0} commontests.go:142: Got 1 recordsets: [0xc0002db470] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002db640} route53_test.go:93: Got zone list: [0xc0002db670] route53_test.go:98: Got at least 1 zone in list:&{0xc0002b3110 0xc0000125a0} commontests.go:192: Successfully added resource record set: {0xc000110c80 0xc0000125c0} commontests.go:70: Correctly replaced resource record {0xc000110c80 0xc0000125c0} -> {0xc000110d80 0xc000012610} commontests.go:142: Got 1 recordsets: [0xc0002db980] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002dbba0} route53_test.go:93: Got zone list: [0xc0002dbbd0] route53_test.go:98: Got at least 1 zone in list:&{0xc0002b3110 0xc0000126d0} commontests.go:192: Successfully added resource record set: {0xc000111000 0xc0000126f0} commontests.go:142: Got 2 recordsets: [0xc0002dbe40 0xc0002dbe60] commontests.go:142: Got 2 recordsets: [0xc0002dbed0 0xc0002dbef0] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002ea090} route53_test.go:93: Got zone list: [0xc0002ea0c0] route53_test.go:98: Got at least 1 zone in list:&{0xc0002b3110 0xc0000127f0} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53 0.060s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53/stubs [no test files] === RUN TestZonesList --- PASS: TestZonesList (0.00s) === RUN TestAdd --- PASS: TestAdd (0.00s) === RUN TestRemove --- PASS: TestRemove (0.00s) === RUN TestNewZone --- PASS: TestNewZone (0.00s) === RUN TestNewResourceRecordSet --- PASS: TestNewResourceRecordSet (0.00s) === RUN TestResourceRecordChangeset --- PASS: TestResourceRecordChangeset (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/do 0.030s === RUN TestZonesList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000012018 0xc000522080} clouddns_test.go:71: Got zone list: [0xc000134390] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000524900 0xc000134360} --- PASS: TestZonesList (0.00s) === RUN TestZonesID clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000012018 0xc000522180} clouddns_test.go:71: Got zone list: [0xc000134438] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000524900 0xc000134408} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess clouddns_test.go:131: Getting zones clouddns_test.go:58: Got zones {0xc000012018 0xc000522280} clouddns_test.go:133: Got zones, making new Zone clouddns_test.go:149: Successfully added managed DNS zone: &{0xc000524a40 0xc0001344e0} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000012018 0xc000522340} clouddns_test.go:71: Got zone list: [0xc000134588] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000524900 0xc000134558} clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000012018 0xc000522460} clouddns_test.go:71: Got zone list: [0xc000134810] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000524900 0xc0001347e0} clouddns_test.go:165: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000012018 0xc0005225a0} clouddns_test.go:71: Got zone list: [0xc000134960] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000524900 0xc000134930} clouddns_test.go:176: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:96: Got 1 recordsets: [<(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000012018 0xc0005230a0} clouddns_test.go:71: Got zone list: [0xc000134f48] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000524900 0xc000134f00} clouddns_test.go:198: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:205: Correctly failed to add duplicate resource record <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>: attempt to insert duplicate rrset {www11.example.com [10.10.10.10 169.20.20.20] 180 A} --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000012018 0xc000523200} clouddns_test.go:71: Got zone list: [0xc000135488] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000524900 0xc000135458} clouddns_test.go:223: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000012018 0xc000523340} clouddns_test.go:71: Got zone list: [0xc0001355d8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000524900 0xc0001355a8} clouddns_test.go:241: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000012018 0xc0005234a0} clouddns_test.go:71: Got zone list: [0xc000135788] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000524900 0xc000135758} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:45: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000012018 0xc000523600} clouddns_test.go:71: Got zone list: [0xc000135cb0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000524900 0xc000135c50} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:70: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000012018 0xc000523780} clouddns_test.go:71: Got zone list: [0xc000576300] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000524900 0xc000576288} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000012018 0xc000523960} clouddns_test.go:71: Got zone list: [0xc000576a50] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000524900 0xc000576a08} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns 0.130s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/interfaces [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/stubs [no test files] === RUN TestContract --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/openstack/designate 0.034s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/rrstype [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/tests [no test files] ? k8s.io/kops/examples/kops-api-example [no test files] === RUN TestBootstarapBuilder_Simple --- PASS: TestBootstarapBuilder_Simple (0.00s) PASS ok k8s.io/kops/nodeup/pkg/bootstrap 0.170s === RUN TestBuildAzure --- PASS: TestBuildAzure (0.00s) === RUN TestBuildAWSCustomNodeIPFamilies --- PASS: TestBuildAWSCustomNodeIPFamilies (0.00s) === RUN TestContainerdBuilder_Docker_19_03_13 I0920 05:49:02.954530 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.954795 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:02.954856 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:02.954907 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:02.954982 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:02.955032 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:02.955056 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:02.955097 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:02.955137 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:02.966771 9448 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_13 (0.02s) === RUN TestContainerdBuilder_Docker_19_03_14 I0920 05:49:02.971497 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.971690 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:02.971727 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:02.971767 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:02.971802 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:02.971833 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:02.971855 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:02.971882 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:02.971904 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:02.973232 9448 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_14 (0.01s) === RUN TestContainerdBuilder_Simple I0920 05:49:02.977744 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.977921 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:02.977961 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:02.978000 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:02.978037 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:02.978068 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:02.978090 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:02.978125 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:02.978145 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:02.979476 9448 zones.go:93] ListHostedZonesPages { } I0920 05:49:02.981531 9448 assetstore.go:108] Matching assets for "^bin/(containerd|ctr)": I0920 05:49:02.981552 9448 assetstore.go:111] bin/containerd I0920 05:49:02.981564 9448 assetstore.go:111] bin/containerd-shim I0920 05:49:02.981574 9448 assetstore.go:111] bin/containerd-shim-runc-v1 I0920 05:49:02.981583 9448 assetstore.go:111] bin/containerd-shim-runc-v2 I0920 05:49:02.981593 9448 assetstore.go:111] bin/containerd-stress I0920 05:49:02.981610 9448 assetstore.go:111] bin/ctr I0920 05:49:02.981684 9448 assetstore.go:108] Matching assets for "/runc\\.(amd64|arm64)$": I0920 05:49:02.981702 9448 assetstore.go:111] https://github.com/opencontainers/runc/releases/download/v1.1.0/runc.amd64 --- PASS: TestContainerdBuilder_Simple (0.01s) === RUN TestContainerdBuilder_Flatcar I0920 05:49:02.986288 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.986508 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:02.986558 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:02.986590 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:02.986633 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:02.986677 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:02.986698 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:02.986725 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:02.986744 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:02.988016 9448 zones.go:93] ListHostedZonesPages { } I0920 05:49:02.989215 9448 containerd.go:58] Detected Flatcar; won't install containerd --- PASS: TestContainerdBuilder_Flatcar (0.00s) === RUN TestContainerdBuilder_SkipInstall I0920 05:49:02.990710 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.990861 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:02.990890 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:02.990911 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:02.990934 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:02.990995 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:02.991015 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:02.991033 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:02.991049 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:02.991818 9448 zones.go:93] ListHostedZonesPages { } I0920 05:49:02.992950 9448 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestContainerdBuilder_SkipInstall (0.00s) === RUN TestContainerdBuilder_BuildFlags --- PASS: TestContainerdBuilder_BuildFlags (0.00s) === RUN TestContainerdConfig --- PASS: TestContainerdConfig (0.00s) === RUN TestAppendGPURuntimeContainerdConfig --- PASS: TestAppendGPURuntimeContainerdConfig (0.00s) === RUN TestDockerBuilder_19_03_11 I0920 05:49:02.993516 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.993623 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:02.993641 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:02.993670 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:02.993689 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:02.993721 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:02.993734 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:02.993748 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:02.993759 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:02.995228 9448 zones.go:93] ListHostedZonesPages { } I0920 05:49:02.997816 9448 assetstore.go:108] Matching assets for "^docker/": I0920 05:49:02.997843 9448 assetstore.go:111] docker/containerd I0920 05:49:02.997851 9448 assetstore.go:111] docker/containerd-shim I0920 05:49:02.997857 9448 assetstore.go:111] docker/ctr I0920 05:49:02.997862 9448 assetstore.go:111] docker/docker I0920 05:49:02.997868 9448 assetstore.go:111] docker/docker-init I0920 05:49:02.997874 9448 assetstore.go:111] docker/docker-proxy I0920 05:49:02.997879 9448 assetstore.go:111] docker/dockerd I0920 05:49:02.997884 9448 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_19_03_11 (0.01s) === RUN TestDockerBuilder_LogFlags I0920 05:49:03.000359 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.000504 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.000533 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.000561 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.000587 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.000632 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.000646 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.000666 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.000681 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.001682 9448 zones.go:93] ListHostedZonesPages { } I0920 05:49:03.003020 9448 assetstore.go:108] Matching assets for "^docker/": I0920 05:49:03.003036 9448 assetstore.go:111] docker/containerd I0920 05:49:03.003043 9448 assetstore.go:111] docker/containerd-shim I0920 05:49:03.003048 9448 assetstore.go:111] docker/ctr I0920 05:49:03.003055 9448 assetstore.go:111] docker/docker I0920 05:49:03.003060 9448 assetstore.go:111] docker/docker-init I0920 05:49:03.003065 9448 assetstore.go:111] docker/docker-proxy I0920 05:49:03.003071 9448 assetstore.go:111] docker/dockerd I0920 05:49:03.003077 9448 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_LogFlags (0.01s) === RUN TestDockerBuilder_SkipInstall I0920 05:49:03.005474 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.005601 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.005637 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.005663 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.005688 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.005729 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.005744 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.005763 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.005777 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.006687 9448 zones.go:93] ListHostedZonesPages { } I0920 05:49:03.007972 9448 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestDockerBuilder_SkipInstall (0.00s) === RUN TestDockerBuilder_HealthCheck I0920 05:49:03.008158 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.008301 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.008329 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.008358 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.008381 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.008417 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.008431 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.008452 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.008475 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.009338 9448 zones.go:93] ListHostedZonesPages { } I0920 05:49:03.010526 9448 assetstore.go:108] Matching assets for "^docker/": I0920 05:49:03.010545 9448 assetstore.go:111] docker/docker I0920 05:49:03.010553 9448 assetstore.go:111] docker/docker-containerd I0920 05:49:03.010558 9448 assetstore.go:111] docker/docker-containerd-ctr I0920 05:49:03.010564 9448 assetstore.go:111] docker/docker-containerd-shim I0920 05:49:03.010570 9448 assetstore.go:111] docker/docker-init I0920 05:49:03.010577 9448 assetstore.go:111] docker/docker-proxy I0920 05:49:03.010582 9448 assetstore.go:111] docker/docker-runc I0920 05:49:03.010588 9448 assetstore.go:111] docker/dockerd --- PASS: TestDockerBuilder_HealthCheck (0.01s) === RUN TestDockerBuilder_BuildFlags --- PASS: TestDockerBuilder_BuildFlags (0.00s) === RUN TestContainerdHooksBuilder I0920 05:49:03.014014 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.014151 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.014181 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.014208 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.014235 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.014257 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.014272 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.014293 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.014306 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.016449 9448 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdHooksBuilder (0.01s) === RUN TestDockerHooksBuilder I0920 05:49:03.019604 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.019921 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.019958 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.020003 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.020049 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.020073 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.020087 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.020108 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.020123 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.021745 9448 zones.go:93] ListHostedZonesPages { } --- PASS: TestDockerHooksBuilder (0.00s) === RUN TestKopsControllerBuilder I0920 05:49:03.023682 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.023796 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.023826 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.023844 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.023864 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.023884 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.023899 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.023915 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.023927 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.025130 9448 zones.go:93] ListHostedZonesPages { } --- PASS: TestKopsControllerBuilder (0.00s) === RUN Test_KubeAPIServer_BuildFlags --- PASS: Test_KubeAPIServer_BuildFlags (0.00s) === RUN TestKubeAPIServerBuilder I0920 05:49:03.029734 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.029881 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.029911 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.029936 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.029963 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.030018 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.030040 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.030059 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.030073 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.031700 9448 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilder (0.01s) === RUN TestDeddicatedAPIServerBuilder I0920 05:49:03.038952 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.039118 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.039150 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.039182 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.039208 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.039230 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.039247 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.039267 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.039282 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.040897 9448 zones.go:93] ListHostedZonesPages { } --- PASS: TestDeddicatedAPIServerBuilder (0.01s) === RUN TestWithoutEtcdEventsAPIServerBuilder I0920 05:49:03.049597 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.049783 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.049824 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.049861 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.049899 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.049931 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.049954 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.050047 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.050076 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.051627 9448 zones.go:93] ListHostedZonesPages { } --- PASS: TestWithoutEtcdEventsAPIServerBuilder (0.01s) === RUN TestAwsIamAuthenticator I0920 05:49:03.056546 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.056714 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.056742 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.056765 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.056790 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.056810 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.056825 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.056844 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.056857 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.058296 9448 zones.go:93] ListHostedZonesPages { } --- PASS: TestAwsIamAuthenticator (0.01s) === RUN TestKubeAPIServerBuilderAMD64 I0920 05:49:03.066073 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.066257 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.066293 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.066326 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.066360 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.066390 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.066410 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.066438 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.066457 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.068458 9448 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderAMD64 (0.82s) === RUN TestKubeAPIServerBuilderARM64 I0920 05:49:03.882939 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.883088 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.883122 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.883149 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.883177 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.883200 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.883215 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.883237 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.883252 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.884845 9448 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderARM64 (0.08s) === RUN TestKubeControllerManagerBuilder I0920 05:49:03.958246 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.958464 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.958512 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.958574 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.958616 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.958654 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.958678 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.958719 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.958745 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.961302 9448 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilder (0.01s) === RUN TestKubeControllerManagerBuilderAMD64 I0920 05:49:03.970530 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:03.970720 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:03.970758 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:03.970791 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:03.970824 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:03.970856 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:03.970875 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:03.970899 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:03.970918 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:03.972973 9448 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderAMD64 (0.08s) === RUN TestKubeControllerManagerBuilderARM64 I0920 05:49:04.047478 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:04.047709 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:04.047761 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:04.047812 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:04.047864 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:04.047909 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:04.047956 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:04.048003 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:04.048032 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:04.051198 9448 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderARM64 (0.08s) === RUN TestKubeProxyBuilder_buildPod === RUN TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 --- PASS: TestKubeProxyBuilder_buildPod (0.00s) --- PASS: TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 (0.00s) === RUN TestKubeProxyBuilder I0920 05:49:04.123279 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:04.123476 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:04.123524 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:04.123569 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:04.123620 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:04.123650 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:04.123668 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:04.123694 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:04.123712 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:04.125731 9448 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilder (0.01s) === RUN TestKubeProxyBuilderAMD64 I0920 05:49:04.131871 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:04.132104 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:04.132155 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:04.132218 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:04.132263 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:04.132304 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:04.132330 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:04.132366 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:04.132393 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:04.135115 9448 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderAMD64 (0.07s) === RUN TestKubeProxyBuilderARM64 I0920 05:49:04.206092 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:04.206303 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:04.206359 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:04.206405 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:04.206451 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:04.206489 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:04.206519 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:04.206557 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:04.206580 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:04.209481 9448 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderARM64 (0.07s) === RUN TestParseDefault --- PASS: TestParseDefault (0.00s) === RUN TestParse --- PASS: TestParse (0.00s) === RUN TestKubeSchedulerBuilder I0920 05:49:04.276233 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:04.276378 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:04.276406 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:04.276427 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:04.276454 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:04.276476 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:04.276491 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:04.276510 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:04.276532 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:04.277978 9448 zones.go:93] ListHostedZonesPages { } W0920 05:49:04.280073 9448 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilder (0.01s) === RUN TestKubeSchedulerBuilderAMD64 I0920 05:49:04.281445 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:04.281547 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:04.281568 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:04.281586 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:04.281618 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:04.281637 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:04.281650 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:04.281668 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:04.281678 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:04.283031 9448 zones.go:93] ListHostedZonesPages { } W0920 05:49:04.359051 9448 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilderAMD64 (0.08s) === RUN TestKubeSchedulerBuilderARM64 I0920 05:49:04.361847 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:04.362091 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:04.362143 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:04.362194 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:04.362242 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:04.362291 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:04.362320 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:04.362367 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:04.362393 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:04.365359 9448 zones.go:93] ListHostedZonesPages { } W0920 05:49:04.433801 9448 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilderARM64 (0.07s) === RUN TestKubectlBuilder I0920 05:49:04.435881 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:04.436049 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:04.436086 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:04.436119 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:04.436159 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:04.436191 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:04.436209 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:04.436234 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:04.436252 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:04.438143 9448 zones.go:93] ListHostedZonesPages { } I0920 05:49:04.439988 9448 assetstore.go:159] Resolved asset kubectl: to /path/to/kubectl/asset --- PASS: TestKubectlBuilder (0.01s) === RUN Test_InstanceGroupKubeletMerge --- PASS: Test_InstanceGroupKubeletMerge (0.00s) === RUN TestTaintsApplied --- PASS: TestTaintsApplied (0.00s) === RUN Test_RunKubeletBuilder I0920 05:49:04.442089 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:04.442237 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:04.442294 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:04.442327 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:04.442369 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:04.442396 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:04.442414 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:04.442438 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:04.442457 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:04.443957 9448 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilder (0.01s) === RUN Test_RunKubeletBuilderWarmPool I0920 05:49:04.447842 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:04.447983 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:04.448030 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:04.448058 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:04.448091 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:04.448119 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:04.448136 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:04.448160 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:04.448176 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:04.449530 9448 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilderWarmPool (0.00s) === RUN Test_BuildComponentConfigFile --- PASS: Test_BuildComponentConfigFile (0.00s) === RUN TestProtokubeBuilder I0920 05:49:04.453187 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:04.453354 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:04.453399 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:04.453425 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:04.453454 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:04.453478 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:04.453496 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:04.453517 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:04.453533 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:04.455282 9448 zones.go:93] ListHostedZonesPages { } I0920 05:49:04.456928 9448 assetstore.go:108] Matching assets for "protokube$": I0920 05:49:04.456947 9448 assetstore.go:111] /opt/kops/bin/protokube I0920 05:49:04.456966 9448 assetstore.go:129] Found single matching asset for expr "protokube$": "protokube" I0920 05:49:04.456989 9448 assetstore.go:108] Matching assets for "channels$": I0920 05:49:04.456998 9448 assetstore.go:111] /opt/kops/bin/channels I0920 05:49:04.457006 9448 assetstore.go:129] Found single matching asset for expr "channels$": "channels" --- PASS: TestProtokubeBuilder (0.01s) === RUN TestSecretBuilder I0920 05:49:04.458503 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:04.458657 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:04.458689 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:04.458717 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:04.458747 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:04.458772 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:04.458790 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:04.458814 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:04.458829 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:04.460514 9448 zones.go:93] ListHostedZonesPages { } --- PASS: TestSecretBuilder (0.00s) === RUN TestUpdateServiceBuilderAutomaticUpgrade I0920 05:49:04.462642 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:04.462843 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:04.462883 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:04.462917 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:04.462949 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:04.462982 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:04.462999 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:04.463022 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:04.463041 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:04.464630 9448 zones.go:93] ListHostedZonesPages { } I0920 05:49:04.466136 9448 update_service.go:96] Detected OS {deb ubuntu focal 20.04}; installing unattended-upgrades package --- PASS: TestUpdateServiceBuilderAutomaticUpgrade (0.00s) === RUN TestUpdateServiceBuilderExternal I0920 05:49:04.466578 9448 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:04.466741 9448 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:04.466770 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:04.466795 9448 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:04.466821 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:04.466843 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:04.466859 9448 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:04.466881 9448 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:04.466896 9448 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:04.469647 9448 zones.go:93] ListHostedZonesPages { } I0920 05:49:04.470937 9448 update_service.go:91] UpdatePolicy requests automatic updates; skipping installation of package "unattended-upgrades" --- PASS: TestUpdateServiceBuilderExternal (0.00s) PASS ok k8s.io/kops/nodeup/pkg/model 1.614s ? k8s.io/kops/nodeup/pkg/model/dns [no test files] ? k8s.io/kops/nodeup/pkg/model/networking [no test files] ? k8s.io/kops/nodeup/pkg/model/resources [no test files] ? k8s.io/kops/pkg/acls [no test files] ? k8s.io/kops/pkg/acls/gce [no test files] === RUN Test_Strategy --- PASS: Test_Strategy (0.00s) === RUN Test_In_StateStore --- PASS: Test_In_StateStore (0.00s) PASS ok k8s.io/kops/pkg/acls/s3 0.072s === RUN TestWarmPoolSpec_IsEnabled === RUN TestWarmPoolSpec_IsEnabled/nil === RUN TestWarmPoolSpec_IsEnabled/empty === RUN TestWarmPoolSpec_IsEnabled/1 === RUN TestWarmPoolSpec_IsEnabled/0 --- PASS: TestWarmPoolSpec_IsEnabled (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/nil (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/empty (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/1 (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/0 (0.00s) === RUN TestWarmPoolSpec_ResolveDefaults === RUN TestWarmPoolSpec_ResolveDefaults/MinSize === RUN TestWarmPoolSpec_ResolveDefaults/MaxSize === RUN TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook --- PASS: TestWarmPoolSpec_ResolveDefaults (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MinSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MaxSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook (0.00s) === RUN Test_ParseInstanceGroupRole --- PASS: Test_ParseInstanceGroupRole (0.00s) === RUN TestParseConfigYAML === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{}" --- PASS: TestParseConfigYAML (0.01s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" (0.01s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{}" (0.00s) === RUN TestWeaveParseConfigYAML === RUN TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" === RUN TestWeaveParseConfigYAML/"networking:_{}" --- PASS: TestWeaveParseConfigYAML (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{}" (0.00s) === RUN Test_SemverOrdering --- PASS: Test_SemverOrdering (0.00s) PASS ok k8s.io/kops/pkg/apis/kops 0.135s === RUN TestRoundTripTypes --- PASS: TestRoundTripTypes (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/install 0.062s === RUN TestUseCiliumEtcd --- PASS: TestUseCiliumEtcd (0.00s) === RUN Test_FindSubnet --- PASS: Test_FindSubnet (0.00s) === RUN Test_FindZonesForInstanceGroup --- PASS: Test_FindZonesForInstanceGroup (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/model 0.086s ? k8s.io/kops/pkg/apis/kops/registry [no test files] === RUN TestGetNodeRole === RUN TestGetNodeRole/RoleNone === RUN TestGetNodeRole/RoleNewerLabel === RUN TestGetNodeRole/RoleOlderLabel --- PASS: TestGetNodeRole (0.00s) --- PASS: TestGetNodeRole/RoleNone (0.00s) --- PASS: TestGetNodeRole/RoleNewerLabel (0.00s) --- PASS: TestGetNodeRole/RoleOlderLabel (0.00s) === RUN Test_ParseKubernetesVersion === RUN Test_ParseKubernetesVersion/1.3.7 === RUN Test_ParseKubernetesVersion/v1.4.0-beta.8 === RUN Test_ParseKubernetesVersion/1.5.0 === RUN Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ === RUN Test_ParseKubernetesVersion/#00 E0920 05:48:45.899760 3388 versions.go:96] unable to parse Kubernetes version "" === RUN Test_ParseKubernetesVersion/abc E0920 05:48:45.900112 3388 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: Test_ParseKubernetesVersion (0.00s) --- PASS: Test_ParseKubernetesVersion/1.3.7 (0.00s) --- PASS: Test_ParseKubernetesVersion/v1.4.0-beta.8 (0.00s) --- PASS: Test_ParseKubernetesVersion/1.5.0 (0.00s) --- PASS: Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ (0.00s) --- PASS: Test_ParseKubernetesVersion/#00 (0.00s) --- PASS: Test_ParseKubernetesVersion/abc (0.00s) === RUN Test_IsKubernetesGTEWithPatch --- PASS: Test_IsKubernetesGTEWithPatch (0.00s) === RUN Test_IsKubernetesGTEWithoutPatch --- PASS: Test_IsKubernetesGTEWithoutPatch (0.00s) === RUN Test_IsKubernetesGTEWithPre --- PASS: Test_IsKubernetesGTEWithPre (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/util 0.013s ? k8s.io/kops/pkg/apis/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha3 [no test files] === RUN TestAWSValidateExternalCloudConfig --- PASS: TestAWSValidateExternalCloudConfig (0.00s) === RUN TestValidateInstanceGroupSpec I0920 05:48:56.030580 7543 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0920 05:48:56.030696 7543 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0920 05:48:56.030736 7543 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0920 05:48:56.030767 7543 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0920 05:48:56.030798 7543 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestValidateInstanceGroupSpec (0.00s) === RUN TestMixedInstancePolicies I0920 05:48:56.031038 7543 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0920 05:48:56.031079 7543 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0920 05:48:56.031107 7543 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0920 05:48:56.031134 7543 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0920 05:48:56.031166 7543 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0920 05:48:56.031194 7543 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0920 05:48:56.031270 7543 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0920 05:48:56.031311 7543 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0920 05:48:56.031347 7543 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0920 05:48:56.031383 7543 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0920 05:48:56.031412 7543 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0920 05:48:56.031444 7543 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0920 05:48:56.031474 7543 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0920 05:48:56.031502 7543 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0920 05:48:56.031527 7543 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0920 05:48:56.031565 7543 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0920 05:48:56.031594 7543 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0920 05:48:56.031630 7543 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0920 05:48:56.031660 7543 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestMixedInstancePolicies (0.00s) === RUN TestInstanceMetadataOptions I0920 05:48:56.031836 7543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } I0920 05:48:56.031896 7543 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } --- PASS: TestInstanceMetadataOptions (0.00s) === RUN TestLoadBalancerSubnets --- PASS: TestLoadBalancerSubnets (0.00s) === RUN TestAWSAuthentication --- PASS: TestAWSAuthentication (0.00s) === RUN TestAWSAdditionalRoutes --- PASS: TestAWSAdditionalRoutes (0.00s) === RUN TestValidEtcdChanges --- PASS: TestValidEtcdChanges (0.00s) === RUN TestValidateInstanceProfile --- PASS: TestValidateInstanceProfile (0.00s) === RUN TestValidMasterInstanceGroup --- PASS: TestValidMasterInstanceGroup (0.00s) === RUN TestValidBootDevice --- PASS: TestValidBootDevice (0.00s) === RUN TestValidNodeLabels --- PASS: TestValidNodeLabels (0.00s) === RUN TestValidateIGCloudLabels --- PASS: TestValidateIGCloudLabels (0.00s) === RUN TestIGCloudLabelIsIGName --- PASS: TestIGCloudLabelIsIGName (0.00s) === RUN TestValidTaints --- PASS: TestValidTaints (0.00s) === RUN TestIGUpdatePolicy === RUN TestIGUpdatePolicy/missing === RUN TestIGUpdatePolicy/automatic === RUN TestIGUpdatePolicy/external === RUN TestIGUpdatePolicy/empty === RUN TestIGUpdatePolicy/unknown --- PASS: TestIGUpdatePolicy (0.00s) --- PASS: TestIGUpdatePolicy/missing (0.00s) --- PASS: TestIGUpdatePolicy/automatic (0.00s) --- PASS: TestIGUpdatePolicy/external (0.00s) --- PASS: TestIGUpdatePolicy/empty (0.00s) --- PASS: TestIGUpdatePolicy/unknown (0.00s) === RUN TestValidInstanceGroup --- PASS: TestValidInstanceGroup (0.00s) === RUN Test_ValidateTopology --- PASS: Test_ValidateTopology (0.00s) === RUN Test_Validate_DNS --- PASS: Test_Validate_DNS (0.00s) === RUN TestValidateCIDR --- PASS: TestValidateCIDR (0.00s) === RUN TestValidateSubnets --- PASS: TestValidateSubnets (0.00s) === RUN TestValidateKubeAPIServer --- PASS: TestValidateKubeAPIServer (0.00s) === RUN Test_Validate_DockerConfig_Storage --- PASS: Test_Validate_DockerConfig_Storage (0.00s) === RUN Test_Validate_Networking_Flannel --- PASS: Test_Validate_Networking_Flannel (0.00s) === RUN Test_Validate_AdditionalPolicies --- PASS: Test_Validate_AdditionalPolicies (0.00s) === RUN Test_Validate_Calico === RUN Test_Validate_Calico/empty_specs === RUN Test_Validate_Calico/positive_Typha_replica_count === RUN Test_Validate_Calico/negative_Typha_replica_count === RUN Test_Validate_Calico/with_etcd_version === RUN Test_Validate_Calico/IPv4_autodetection_method === RUN Test_Validate_Calico/IPv6_autodetection_method === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name === RUN Test_Validate_Calico/AWS_source/destination_checks_off === RUN Test_Validate_Calico/AWS_source/destination_checks_enabled === RUN Test_Validate_Calico/AWS_source/destination_checks_disabled === RUN Test_Validate_Calico/AWS_source/destination_checks_left_as_is === RUN Test_Validate_Calico/encapsulation_none_with_IPv4 === RUN Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 === RUN Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 === RUN Test_Validate_Calico/unknown_Calico_IPIP_mode === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPv6_without_encapsulation --- PASS: Test_Validate_Calico (0.00s) --- PASS: Test_Validate_Calico/empty_specs (0.00s) --- PASS: Test_Validate_Calico/positive_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/negative_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/with_etcd_version (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_off (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_enabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_disabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_left_as_is (0.00s) --- PASS: Test_Validate_Calico/encapsulation_none_with_IPv4 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/unknown_Calico_IPIP_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPv6_without_encapsulation (0.00s) === RUN Test_Validate_Cilium --- PASS: Test_Validate_Cilium (0.00s) === RUN Test_Validate_RollingUpdate --- PASS: Test_Validate_RollingUpdate (0.00s) === RUN Test_Validate_NodeLocalDNS --- PASS: Test_Validate_NodeLocalDNS (0.00s) === RUN Test_Validate_CloudConfiguration === RUN Test_Validate_CloudConfiguration/neither === RUN Test_Validate_CloudConfiguration/all_false === RUN Test_Validate_CloudConfiguration/all_true === RUN Test_Validate_CloudConfiguration/os_false === RUN Test_Validate_CloudConfiguration/os_true === RUN Test_Validate_CloudConfiguration/all_false,_os_false === RUN Test_Validate_CloudConfiguration/all_false,_os_true === RUN Test_Validate_CloudConfiguration/all_true,_os_false === RUN Test_Validate_CloudConfiguration/all_true,_os_true --- PASS: Test_Validate_CloudConfiguration (0.00s) --- PASS: Test_Validate_CloudConfiguration/neither (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_true (0.00s) === RUN TestValidateSAExternalPermissions === RUN TestValidateSAExternalPermissions/Duplicate_SA === RUN TestValidateSAExternalPermissions/Missing_permissions === RUN TestValidateSAExternalPermissions/Setting_both_arn_and_inline === RUN TestValidateSAExternalPermissions/Empty_SA_name === RUN TestValidateSAExternalPermissions/Empty_SA_namespace --- PASS: TestValidateSAExternalPermissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Duplicate_SA (0.00s) --- PASS: TestValidateSAExternalPermissions/Missing_permissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Setting_both_arn_and_inline (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_name (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_namespace (0.00s) === RUN Test_Validate_Nvidia_Cluster --- PASS: Test_Validate_Nvidia_Cluster (0.00s) === RUN Test_Validate_Nvidia_Ig --- PASS: Test_Validate_Nvidia_Ig (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/validation 0.144s ? k8s.io/kops/pkg/apis/nodeup [no test files] ? k8s.io/kops/pkg/applylib [no test files] === RUN TestApplySet I0920 05:48:47.514473 4941 apiserver.go:85] kubeapiserver request: GET /api?timeout=32s I0920 05:48:47.515460 4941 apiserver.go:85] kubeapiserver request: GET /apis?timeout=32s I0920 05:48:47.516467 4941 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I0920 05:48:47.517121 4941 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I0920 05:48:47.518692 4941 apiserver.go:85] kubeapiserver request: GET /api?timeout=32s I0920 05:48:47.519156 4941 apiserver.go:85] kubeapiserver request: GET /apis?timeout=32s I0920 05:48:47.519674 4941 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I0920 05:48:47.519879 4941 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I0920 05:48:47.531840 4941 apiserver.go:85] kubeapiserver request: PATCH /api/v1/namespaces/test-applyset?fieldManager=kops&force=true I0920 05:48:47.531937 4941 patchresource.go:57] patch request "{\"apiVersion\":\"v1\",\"kind\":\"Namespace\",\"metadata\":{\"name\":\"test-applyset\"}}" I0920 05:48:47.532227 4941 health.go:33] status conditions not found for /v1, Kind=Namespace I0920 05:48:47.532416 4941 apiserver.go:85] kubeapiserver request: PATCH /api/v1/namespaces/test-applyset/configmaps/foo?fieldManager=kops&force=true I0920 05:48:47.532469 4941 patchresource.go:57] patch request "{\"apiVersion\":\"v1\",\"data\":{\"foo\":\"bar\"},\"kind\":\"ConfigMap\",\"metadata\":{\"name\":\"foo\",\"namespace\":\"test-applyset\"}}" I0920 05:48:47.532681 4941 health.go:33] status conditions not found for /v1, Kind=ConfigMap I0920 05:48:47.532854 4941 apiserver.go:85] kubeapiserver request: GET /api/v1/namespaces/test-applyset I0920 05:48:47.532915 4941 getresource.go:64] returning {"apiVersion":"v1","kind":"Namespace","metadata":{"labels":{"kubernetes.io/metadata.name":"test-applyset"},"name":"test-applyset"},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Active"}} I0920 05:48:47.540703 4941 apiserver.go:85] kubeapiserver request: GET /api/v1/namespaces/test-applyset/configmaps/foo I0920 05:48:47.540795 4941 getresource.go:64] returning {"apiVersion":"v1","data":{"foo":"bar"},"kind":"ConfigMap","metadata":{"name":"foo","namespace":"test-applyset"}} --- PASS: TestApplySet (0.03s) PASS ok k8s.io/kops/pkg/applylib/applyset 0.098s ? k8s.io/kops/pkg/applylib/mocks [no test files] ? k8s.io/kops/pkg/applylib/mocks/mockkubeapiserver [no test files] === RUN TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub W0920 05:48:46.140924 3765 builder.go:231] failed to digest image "proxy.example.com/weaveworks/weave-kube" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub (0.01s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub W0920 05:48:46.142569 3765 builder.go:231] failed to digest image "proxy.example.com/debian" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL W0920 05:48:46.143628 3765 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL W0920 05:48:46.144609 3765 builder.go:231] failed to digest image "proxy.example.com/google_containers/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags W0920 05:48:46.145529 3765 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags (0.00s) === RUN TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges W0920 05:48:46.146490 3765 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" W0920 05:48:46.147290 3765 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges (0.00s) === RUN TestRemapEmptySection --- PASS: TestRemapEmptySection (0.00s) === RUN Test_BuildVFSPath E0920 05:48:46.147940 3765 copyfile.go:205] Unable to determine VFS path from supplied URL: https://foo/k8s-for-greeks-kops/kubernetes-release/release/v1.7.2/bin/linux/amd64/kubectl E0920 05:48:46.147959 3765 copyfile.go:206] S3, Google Cloud Storage, and File Paths are supported. E0920 05:48:46.147968 3765 copyfile.go:207] For S3, please make sure that the supplied file repository URL adhere to S3 naming conventions, https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region. E0920 05:48:46.147976 3765 copyfile.go:208] For GCS, please make sure that the supplied file repository URL adheres to https://storage.googleapis.com/ --- PASS: Test_BuildVFSPath (0.00s) PASS ok k8s.io/kops/pkg/assets 0.089s === RUN TestBackoff --- PASS: TestBackoff (0.00s) PASS ok k8s.io/kops/pkg/backoff 0.034s ? k8s.io/kops/pkg/bootstrap [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/simple [no test files] ? k8s.io/kops/pkg/client/simple/api [no test files] === RUN TestSSHCredentialStoreOnConfigBase --- PASS: TestSSHCredentialStoreOnConfigBase (0.00s) === RUN TestSSHCredentialStoreOnOwnCFS --- PASS: TestSSHCredentialStoreOnOwnCFS (0.00s) PASS ok k8s.io/kops/pkg/client/simple/vfsclientset 0.139s === RUN TestToAzureVMName === RUN TestToAzureVMName/test_case_0 === RUN TestToAzureVMName/test_case_1 --- PASS: TestToAzureVMName (0.00s) --- PASS: TestToAzureVMName/test_case_0 (0.00s) --- PASS: TestToAzureVMName/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/cloudinstances 0.052s ? k8s.io/kops/pkg/clouds [no test files] ? k8s.io/kops/pkg/clusteraddons [no test files] === RUN TestSetClusterBadInput --- PASS: TestSetClusterBadInput (0.00s) === RUN TestSetClusterFields --- PASS: TestSetClusterFields (0.01s) === RUN TestSetCiliumFields --- PASS: TestSetCiliumFields (0.00s) === RUN TestSetInstanceGroupsBadInput --- PASS: TestSetInstanceGroupsBadInput (0.00s) === RUN TestSetInstanceGroupsFields --- PASS: TestSetInstanceGroupsFields (0.00s) === RUN TestUnsetClusterBadInput --- PASS: TestUnsetClusterBadInput (0.00s) === RUN TestUnsetClusterFields --- PASS: TestUnsetClusterFields (0.01s) === RUN TestUnsetCiliumFields --- PASS: TestUnsetCiliumFields (0.00s) === RUN TestUnsetInstanceGroupsBadInput --- PASS: TestUnsetInstanceGroupsBadInput (0.00s) === RUN TestUnsetInstanceGroupsFields --- PASS: TestUnsetInstanceGroupsFields (0.00s) PASS ok k8s.io/kops/pkg/commands 0.204s ? k8s.io/kops/pkg/commands/commandutils [no test files] ? k8s.io/kops/pkg/commands/helpers [no test files] === RUN TestGetStructVal --- PASS: TestGetStructVal (0.00s) === RUN TestWrongStructField --- PASS: TestWrongStructField (0.00s) PASS ok k8s.io/kops/pkg/configbuilder 0.040s ? k8s.io/kops/pkg/configserver [no test files] === RUN Test_Diff_1 --- PASS: Test_Diff_1 (0.00s) === RUN Test_Diff_2 --- PASS: Test_Diff_2 (0.00s) === RUN Test_Diff_3 --- PASS: Test_Diff_3 (0.00s) === RUN Test_Diff_ChangedLine --- PASS: Test_Diff_ChangedLine (0.00s) === RUN Test_Diff_4 --- PASS: Test_Diff_4 (0.00s) PASS ok k8s.io/kops/pkg/diff 0.044s === RUN TestIsGossipHostname --- PASS: TestIsGossipHostname (0.00s) PASS ok k8s.io/kops/pkg/dns 0.043s ? k8s.io/kops/pkg/dump [no test files] === RUN TestHasExtraFields --- PASS: TestHasExtraFields (0.03s) PASS ok k8s.io/kops/pkg/edit 0.093s === RUN TestFlagToFalse I0920 05:48:46.324295 4010 featureflag_test.go:32] Created flag Unittest1 I0920 05:48:46.324352 4010 featureflag.go:160] FeatureFlag "UnitTest1"=false I0920 05:48:46.324368 4010 featureflag.go:160] FeatureFlag "UnitTest1"=true --- PASS: TestFlagToFalse (0.00s) === RUN TestSetenv I0920 05:48:46.324440 4010 featureflag.go:160] FeatureFlag "UnitTest2"=false --- PASS: TestSetenv (0.00s) PASS ok k8s.io/kops/pkg/featureflag 0.033s === RUN TestBuildKCMFlags --- PASS: TestBuildKCMFlags (0.00s) === RUN TestKubeletConfigSpec --- PASS: TestKubeletConfigSpec (0.00s) === RUN TestBuildAPIServerFlags --- PASS: TestBuildAPIServerFlags (0.00s) PASS ok k8s.io/kops/pkg/flagbuilder 0.041s === RUN TestRenderInstanceGroupZones W0920 05:48:48.697739 5532 instancegroup.go:42] error fetching zones for instancegroup: cannot find subnet "badsubnet" (declared in instance group "", not found in cluster) --- PASS: TestRenderInstanceGroupZones (0.00s) === RUN TestRenderInstanceGroupSubnets --- PASS: TestRenderInstanceGroupSubnets (0.00s) PASS ok k8s.io/kops/pkg/formatter 0.074s === RUN TestWarmPoolOnlyRoll I0920 05:49:13.817384 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.817461 9789 instancegroups.go:505] Cluster validated. I0920 05:49:13.817479 9789 instancegroups.go:112] deleting warm pool instance "node-1" --- PASS: TestWarmPoolOnlyRoll (0.00s) === RUN TestRollingUpdateDisabledSurgeOS rollingupdate_os_test.go:99: Failing in new release PR when build is not yet published --- SKIP: TestRollingUpdateDisabledSurgeOS (0.00s) === RUN TestRollingUpdateAllNeedUpdate I0920 05:49:13.817875 9789 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0920 05:49:13.817910 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:13.819094 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.819120 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:13.824310 9789 instancegroups.go:505] Cluster validated. I0920 05:49:13.824348 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.824369 9789 instancegroups.go:505] Cluster validated. I0920 05:49:13.824392 9789 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I0920 05:49:13.826959 9789 instancegroups.go:402] Draining the node: "master-1a.local". I0920 05:49:13.827640 9789 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0920 05:49:13.827669 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:13.828845 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.828891 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:13.834162 9789 instancegroups.go:505] Cluster validated. I0920 05:49:13.834200 9789 instancegroups.go:402] Draining the node: "master-1b.local". I0920 05:49:13.834863 9789 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0920 05:49:13.834891 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:13.836085 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.836120 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:13.841384 9789 instancegroups.go:505] Cluster validated. I0920 05:49:13.841417 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.841434 9789 instancegroups.go:505] Cluster validated. I0920 05:49:13.841460 9789 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I0920 05:49:13.842391 9789 instancegroups.go:402] Draining the node: "node-1a.local". I0920 05:49:13.843076 9789 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0920 05:49:13.843107 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:13.844317 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.844368 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:13.849641 9789 instancegroups.go:505] Cluster validated. I0920 05:49:13.849693 9789 instancegroups.go:402] Draining the node: "node-1b.local". I0920 05:49:13.850542 9789 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0920 05:49:13.850578 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:13.851790 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.851814 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:13.857910 9789 instancegroups.go:505] Cluster validated. I0920 05:49:13.857964 9789 instancegroups.go:402] Draining the node: "node-1c.local". I0920 05:49:13.858850 9789 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0920 05:49:13.858883 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:13.860065 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.860108 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:13.865365 9789 instancegroups.go:505] Cluster validated. I0920 05:49:13.865416 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.865439 9789 instancegroups.go:505] Cluster validated. I0920 05:49:13.865455 9789 instancegroups.go:313] Tainting 3 nodes in "node-2" instancegroup. I0920 05:49:13.866895 9789 instancegroups.go:402] Draining the node: "node-2a.local". I0920 05:49:13.867650 9789 instancegroups.go:593] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0920 05:49:13.867697 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:13.868924 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.868948 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:13.874206 9789 instancegroups.go:505] Cluster validated. I0920 05:49:13.874276 9789 instancegroups.go:402] Draining the node: "node-2b.local". I0920 05:49:13.875646 9789 instancegroups.go:593] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0920 05:49:13.875698 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:13.876905 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.876936 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:13.882179 9789 instancegroups.go:505] Cluster validated. I0920 05:49:13.882257 9789 instancegroups.go:402] Draining the node: "node-2c.local". I0920 05:49:13.883688 9789 instancegroups.go:593] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0920 05:49:13.883745 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:13.884985 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.885031 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:13.890280 9789 instancegroups.go:505] Cluster validated. I0920 05:49:13.890320 9789 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdate (0.07s) === RUN TestRollingUpdateAllNeedUpdateCloudonly I0920 05:49:13.891092 9789 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0920 05:49:13.891144 9789 instancegroups.go:438] waiting for 1ms after terminating instance W0920 05:49:13.892334 9789 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W0920 05:49:13.892405 9789 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W0920 05:49:13.892446 9789 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I0920 05:49:13.892464 9789 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0920 05:49:13.892492 9789 instancegroups.go:438] waiting for 1ms after terminating instance W0920 05:49:13.893650 9789 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W0920 05:49:13.893719 9789 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I0920 05:49:13.893746 9789 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0920 05:49:13.893791 9789 instancegroups.go:438] waiting for 1ms after terminating instance W0920 05:49:13.894966 9789 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W0920 05:49:13.895015 9789 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W0920 05:49:13.895084 9789 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I0920 05:49:13.895108 9789 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0920 05:49:13.895146 9789 instancegroups.go:438] waiting for 1ms after terminating instance W0920 05:49:13.896378 9789 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W0920 05:49:13.896461 9789 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I0920 05:49:13.896490 9789 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0920 05:49:13.896555 9789 instancegroups.go:438] waiting for 1ms after terminating instance W0920 05:49:13.897806 9789 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W0920 05:49:13.897913 9789 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I0920 05:49:13.897946 9789 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0920 05:49:13.898002 9789 instancegroups.go:438] waiting for 1ms after terminating instance W0920 05:49:13.899190 9789 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W0920 05:49:13.899226 9789 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W0920 05:49:13.899308 9789 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I0920 05:49:13.899336 9789 instancegroups.go:593] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0920 05:49:13.899379 9789 instancegroups.go:438] waiting for 1ms after terminating instance W0920 05:49:13.900552 9789 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W0920 05:49:13.900636 9789 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I0920 05:49:13.900667 9789 instancegroups.go:593] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0920 05:49:13.900710 9789 instancegroups.go:438] waiting for 1ms after terminating instance W0920 05:49:13.901880 9789 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W0920 05:49:13.901944 9789 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I0920 05:49:13.901968 9789 instancegroups.go:593] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0920 05:49:13.902011 9789 instancegroups.go:438] waiting for 1ms after terminating instance W0920 05:49:13.903190 9789 instancegroups.go:470] Not validating cluster as cloudonly flag is set. I0920 05:49:13.903235 9789 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateCloudonly (0.01s) === RUN TestRollingUpdateAllNeedUpdateNoFailOnValidate I0920 05:49:13.903714 9789 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0920 05:49:13.903758 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:13.904927 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.904978 9789 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W0920 05:49:13.905010 9789 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0920 05:49:13.905095 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.905136 9789 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W0920 05:49:13.905162 9789 instancegroups.go:481] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0920 05:49:13.905187 9789 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I0920 05:49:13.906423 9789 instancegroups.go:402] Draining the node: "master-1a.local". I0920 05:49:13.907776 9789 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0920 05:49:13.907830 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:13.909030 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.909093 9789 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W0920 05:49:13.909127 9789 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0920 05:49:13.909193 9789 instancegroups.go:402] Draining the node: "master-1b.local". I0920 05:49:13.910271 9789 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0920 05:49:13.910325 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:13.911496 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.911544 9789 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W0920 05:49:13.911573 9789 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0920 05:49:13.911611 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.911647 9789 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W0920 05:49:13.911672 9789 instancegroups.go:481] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0920 05:49:13.911693 9789 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I0920 05:49:13.913223 9789 instancegroups.go:402] Draining the node: "node-1a.local". I0920 05:49:13.914516 9789 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0920 05:49:13.914567 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:13.915749 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.915803 9789 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W0920 05:49:13.915836 9789 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0920 05:49:13.915904 9789 instancegroups.go:402] Draining the node: "node-1b.local". I0920 05:49:13.916863 9789 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0920 05:49:13.916915 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:13.918094 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.918140 9789 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W0920 05:49:13.918167 9789 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0920 05:49:13.918225 9789 instancegroups.go:402] Draining the node: "node-1c.local". I0920 05:49:13.919489 9789 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0920 05:49:13.919538 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:13.920727 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.920782 9789 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W0920 05:49:13.920814 9789 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0920 05:49:13.920842 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.920878 9789 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W0920 05:49:13.920913 9789 instancegroups.go:481] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0920 05:49:13.920938 9789 instancegroups.go:313] Tainting 3 nodes in "node-2" instancegroup. I0920 05:49:13.922089 9789 instancegroups.go:402] Draining the node: "node-2a.local". I0920 05:49:13.922967 9789 instancegroups.go:593] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0920 05:49:13.923018 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:13.924184 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.924234 9789 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W0920 05:49:13.924270 9789 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0920 05:49:13.924338 9789 instancegroups.go:402] Draining the node: "node-2b.local". I0920 05:49:13.925684 9789 instancegroups.go:593] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0920 05:49:13.925739 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:13.926913 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.926966 9789 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W0920 05:49:13.926998 9789 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0920 05:49:13.927065 9789 instancegroups.go:402] Draining the node: "node-2c.local". I0920 05:49:13.928017 9789 instancegroups.go:593] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0920 05:49:13.928067 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:13.929241 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.929291 9789 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W0920 05:49:13.929322 9789 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0920 05:49:13.929358 9789 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateNoFailOnValidate (0.03s) === RUN TestRollingUpdateNoneNeedUpdate I0920 05:49:13.929801 9789 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdate (0.00s) === RUN TestRollingUpdateNoneNeedUpdateWithForce I0920 05:49:13.930321 9789 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0920 05:49:13.930371 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:13.931556 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.931601 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:13.936856 9789 instancegroups.go:505] Cluster validated. I0920 05:49:13.936935 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.936957 9789 instancegroups.go:505] Cluster validated. I0920 05:49:13.936971 9789 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I0920 05:49:13.937715 9789 instancegroups.go:402] Draining the node: "master-1a.local". I0920 05:49:13.938537 9789 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0920 05:49:13.938575 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:13.939758 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.939805 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:13.945351 9789 instancegroups.go:505] Cluster validated. I0920 05:49:13.945401 9789 instancegroups.go:402] Draining the node: "master-1b.local". I0920 05:49:13.946393 9789 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0920 05:49:13.946429 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:13.947631 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.947669 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:13.952924 9789 instancegroups.go:505] Cluster validated. I0920 05:49:13.952962 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.952983 9789 instancegroups.go:505] Cluster validated. I0920 05:49:13.952996 9789 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I0920 05:49:13.954224 9789 instancegroups.go:402] Draining the node: "node-1a.local". I0920 05:49:13.955122 9789 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0920 05:49:13.955158 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:13.956372 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.956409 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:13.961710 9789 instancegroups.go:505] Cluster validated. I0920 05:49:13.961771 9789 instancegroups.go:402] Draining the node: "node-1b.local". I0920 05:49:13.962757 9789 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0920 05:49:13.962807 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:13.964009 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.964047 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:13.969316 9789 instancegroups.go:505] Cluster validated. I0920 05:49:13.969367 9789 instancegroups.go:402] Draining the node: "node-1c.local". I0920 05:49:13.970398 9789 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0920 05:49:13.970435 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:13.971646 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.971684 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:13.976926 9789 instancegroups.go:505] Cluster validated. I0920 05:49:13.976957 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.976977 9789 instancegroups.go:505] Cluster validated. I0920 05:49:13.977006 9789 instancegroups.go:313] Tainting 3 nodes in "node-2" instancegroup. I0920 05:49:13.978162 9789 instancegroups.go:402] Draining the node: "node-2a.local". I0920 05:49:13.979080 9789 instancegroups.go:593] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0920 05:49:13.979116 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:13.980318 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.980355 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:13.985637 9789 instancegroups.go:505] Cluster validated. I0920 05:49:13.985689 9789 instancegroups.go:402] Draining the node: "node-2b.local". I0920 05:49:13.986725 9789 instancegroups.go:593] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0920 05:49:13.986769 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:13.987976 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.988019 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:13.993296 9789 instancegroups.go:505] Cluster validated. I0920 05:49:13.993359 9789 instancegroups.go:402] Draining the node: "node-2c.local". I0920 05:49:13.994366 9789 instancegroups.go:593] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0920 05:49:13.994403 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:13.995613 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:13.995649 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.000913 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.000946 9789 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdateWithForce (0.07s) === RUN TestRollingUpdateEmptyGroup I0920 05:49:14.001069 9789 rollingupdate.go:96] Cloud Instance Group length is zero. Not doing a rolling-update. --- PASS: TestRollingUpdateEmptyGroup (0.00s) === RUN TestRollingUpdateUnknownRole --- PASS: TestRollingUpdateUnknownRole (0.00s) === RUN TestRollingUpdateAllNeedUpdateFailsValidation I0920 05:49:14.001627 9789 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0920 05:49:14.001661 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.002862 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.002906 9789 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. E0920 05:49:14.002928 9789 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateFailsValidation (0.00s) === RUN TestRollingUpdateAllNeedUpdateErrorsValidation I0920 05:49:14.003537 9789 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0920 05:49:14.003590 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.004831 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.004866 9789 instancegroups.go:515] Cluster did not validate within deadline: testing validation error. E0920 05:49:14.004888 9789 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateFailsValidation I0920 05:49:14.005184 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.005237 9789 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. E0920 05:49:14.005257 9789 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateNodes1NeedsUpdateFailsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateErrorsValidation I0920 05:49:14.005452 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.005478 9789 instancegroups.go:515] Cluster did not validate within deadline: testing validation error. E0920 05:49:14.005495 9789 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateNodes1NeedsUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneMaster I0920 05:49:14.005788 9789 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0920 05:49:14.005819 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.007018 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.007062 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.012335 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.012396 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.012420 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.012449 9789 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I0920 05:49:14.013297 9789 instancegroups.go:402] Draining the node: "master-1a.local". I0920 05:49:14.014299 9789 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0920 05:49:14.014336 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.015540 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.015588 9789 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. E0920 05:49:14.015609 9789 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneMaster I0920 05:49:14.015990 9789 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0920 05:49:14.016025 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.017238 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.017279 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.022564 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.022629 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.022669 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.022689 9789 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I0920 05:49:14.023796 9789 instancegroups.go:402] Draining the node: "master-1a.local". I0920 05:49:14.025313 9789 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0920 05:49:14.025369 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.027703 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.027761 9789 instancegroups.go:515] Cluster did not validate within deadline: testing validation error. E0920 05:49:14.027794 9789 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateNonRelatedInstanceGroupFailure I0920 05:49:14.028310 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.028351 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.028374 9789 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I0920 05:49:14.029957 9789 instancegroups.go:402] Draining the node: "node-1a.local". I0920 05:49:14.032819 9789 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0920 05:49:14.032861 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.034086 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.034136 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.039340 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.039397 9789 instancegroups.go:402] Draining the node: "node-1b.local". I0920 05:49:14.040332 9789 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0920 05:49:14.040371 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.041603 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.041652 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.046947 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.046997 9789 instancegroups.go:402] Draining the node: "node-1c.local". I0920 05:49:14.047894 9789 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0920 05:49:14.047930 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.049153 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.049198 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.054486 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.054521 9789 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNonRelatedInstanceGroupFailure (0.03s) === RUN TestRollingUpdateRelatedInstanceGroupFailure I0920 05:49:14.054840 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.054872 9789 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. E0920 05:49:14.054892 9789 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateRelatedInstanceGroupFailure (0.00s) === RUN TestRollingUpdateMasterGroupFailure I0920 05:49:14.055118 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.055145 9789 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. E0920 05:49:14.055160 9789 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateMasterGroupFailure (0.00s) === RUN TestRollingUpdateValidationErrorInstanceGroupNil I0920 05:49:14.055430 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.055459 9789 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. E0920 05:49:14.055475 9789 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateValidationErrorInstanceGroupNil (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneNode I0920 05:49:14.055648 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.055676 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.055690 9789 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I0920 05:49:14.056976 9789 instancegroups.go:402] Draining the node: "node-1a.local". I0920 05:49:14.057859 9789 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0920 05:49:14.057894 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.059075 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.059146 9789 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. E0920 05:49:14.059183 9789 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneNode I0920 05:49:14.059609 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.059662 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.059687 9789 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I0920 05:49:14.061806 9789 instancegroups.go:402] Draining the node: "node-1a.local". I0920 05:49:14.063402 9789 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0920 05:49:14.063467 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.064726 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.064767 9789 instancegroups.go:515] Cluster did not validate within deadline: testing validation error. E0920 05:49:14.064789 9789 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneNode (0.01s) === RUN TestRollingUpdateFlappingValidation I0920 05:49:14.065126 9789 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0920 05:49:14.065158 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.066331 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.066400 9789 instancegroups.go:528] Cluster did not pass validation, will retry in "1ms": testing failure. I0920 05:49:14.067667 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.072951 9789 instancegroups.go:528] Cluster did not pass validation, will retry in "1ms": testing failure. I0920 05:49:14.074150 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.079452 9789 instancegroups.go:528] Cluster did not pass validation, will retry in "1ms": testing failure. I0920 05:49:14.080652 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.085981 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.086078 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.086111 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.086131 9789 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I0920 05:49:14.086945 9789 instancegroups.go:402] Draining the node: "master-1a.local". I0920 05:49:14.087762 9789 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0920 05:49:14.087805 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.089003 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.089044 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.094314 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.094389 9789 instancegroups.go:402] Draining the node: "master-1b.local". I0920 05:49:14.095225 9789 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0920 05:49:14.095265 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.096467 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.096522 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.101769 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.101808 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.101835 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.101850 9789 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I0920 05:49:14.102836 9789 instancegroups.go:402] Draining the node: "node-1a.local". I0920 05:49:14.103723 9789 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0920 05:49:14.103762 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.104933 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.104981 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.110243 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.110301 9789 instancegroups.go:402] Draining the node: "node-1b.local". I0920 05:49:14.111147 9789 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0920 05:49:14.111187 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.112381 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.112430 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.117694 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.117753 9789 instancegroups.go:402] Draining the node: "node-1c.local". I0920 05:49:14.118586 9789 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0920 05:49:14.118625 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.119822 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.119877 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.125125 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.125158 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.125185 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.125200 9789 instancegroups.go:313] Tainting 3 nodes in "node-2" instancegroup. I0920 05:49:14.126203 9789 instancegroups.go:402] Draining the node: "node-2a.local". I0920 05:49:14.127003 9789 instancegroups.go:593] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0920 05:49:14.127047 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.128244 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.128296 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.134038 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.134099 9789 instancegroups.go:402] Draining the node: "node-2b.local". I0920 05:49:14.134943 9789 instancegroups.go:593] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0920 05:49:14.134982 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.136158 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.136217 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.141456 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.141515 9789 instancegroups.go:402] Draining the node: "node-2c.local". I0920 05:49:14.142364 9789 instancegroups.go:593] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0920 05:49:14.142402 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.143585 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.143641 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.148964 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.148998 9789 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateFlappingValidation (0.08s) === RUN TestRollingUpdateValidatesAfterBastion I0920 05:49:14.149319 9789 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0920 05:49:14.149354 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.150515 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.150584 9789 instancegroups.go:528] Cluster did not pass validation, will retry in "1ms": testing failure. I0920 05:49:14.151808 9789 instancegroups.go:528] Cluster did not pass validation, will retry in "1ms": testing failure. I0920 05:49:14.152991 9789 instancegroups.go:528] Cluster did not pass validation, will retry in "1ms": testing failure. I0920 05:49:14.154163 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.159430 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.159528 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.159572 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.159604 9789 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I0920 05:49:14.160895 9789 instancegroups.go:402] Draining the node: "master-1a.local". I0920 05:49:14.162442 9789 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0920 05:49:14.162504 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.163712 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.163754 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.168975 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.169056 9789 instancegroups.go:402] Draining the node: "master-1b.local". I0920 05:49:14.170547 9789 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0920 05:49:14.170610 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.171814 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.171856 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.177076 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.177145 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.177186 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.177220 9789 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I0920 05:49:14.179309 9789 instancegroups.go:402] Draining the node: "node-1a.local". I0920 05:49:14.180938 9789 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0920 05:49:14.181001 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.182216 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.182258 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.187474 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.187547 9789 instancegroups.go:402] Draining the node: "node-1b.local". I0920 05:49:14.189101 9789 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0920 05:49:14.189156 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.190347 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.190387 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.196381 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.196441 9789 instancegroups.go:402] Draining the node: "node-1c.local". I0920 05:49:14.197332 9789 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0920 05:49:14.197373 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.198568 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.198603 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.203832 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.203865 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.203886 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.203901 9789 instancegroups.go:313] Tainting 3 nodes in "node-2" instancegroup. I0920 05:49:14.204913 9789 instancegroups.go:402] Draining the node: "node-2a.local". I0920 05:49:14.205803 9789 instancegroups.go:593] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0920 05:49:14.205842 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.207030 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.207066 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.212309 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.212363 9789 instancegroups.go:402] Draining the node: "node-2b.local". I0920 05:49:14.213272 9789 instancegroups.go:593] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0920 05:49:14.213332 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.214504 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.214539 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.219785 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.219839 9789 instancegroups.go:402] Draining the node: "node-2c.local". I0920 05:49:14.220700 9789 instancegroups.go:593] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0920 05:49:14.220740 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.221942 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.221978 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.227202 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.227238 9789 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateValidatesAfterBastion (0.08s) === RUN TestAddAnnotatedNodesToNeedsUpdate --- PASS: TestAddAnnotatedNodesToNeedsUpdate (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateCloudonly --- PASS: TestAddAnnotatedNodesToNeedsUpdateCloudonly (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateNodesMissing --- PASS: TestAddAnnotatedNodesToNeedsUpdateNodesMissing (0.00s) === RUN TestRollingUpdateTaintAllButOneNeedUpdate I0920 05:49:14.227882 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.227909 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.227922 9789 instancegroups.go:313] Tainting 2 nodes in "node-1" instancegroup. I0920 05:49:14.228687 9789 instancegroups.go:402] Draining the node: "node-1a.local". I0920 05:49:14.229471 9789 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0920 05:49:14.229507 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.230700 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.230735 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.236000 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.236051 9789 instancegroups.go:402] Draining the node: "node-1b.local". I0920 05:49:14.236997 9789 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0920 05:49:14.237035 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.238231 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.238265 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.243494 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.243525 9789 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateTaintAllButOneNeedUpdate (0.02s) === RUN TestRollingUpdateMaxSurgeIgnoredForMaster I0920 05:49:14.243796 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.243824 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.243836 9789 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I0920 05:49:14.244516 9789 instancegroups.go:402] Draining the node: "master-1a.local". I0920 05:49:14.245317 9789 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0920 05:49:14.245367 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.246514 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.246543 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.251799 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.251865 9789 instancegroups.go:402] Draining the node: "master-1b.local". I0920 05:49:14.252750 9789 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0920 05:49:14.252794 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.253953 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.253997 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.259265 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.259298 9789 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeIgnoredForMaster (0.02s) === RUN TestRollingUpdateDisabled I0920 05:49:14.259639 9789 instancegroups.go:187] Rolling updates for InstanceGroup bastion-1 are disabled W0920 05:49:14.259685 9789 instancegroups.go:470] Not validating cluster as cloudonly flag is set. I0920 05:49:14.259708 9789 instancegroups.go:187] Rolling updates for InstanceGroup master-1 are disabled W0920 05:49:14.259724 9789 instancegroups.go:470] Not validating cluster as cloudonly flag is set. I0920 05:49:14.259747 9789 instancegroups.go:187] Rolling updates for InstanceGroup node-1 are disabled W0920 05:49:14.259756 9789 instancegroups.go:470] Not validating cluster as cloudonly flag is set. I0920 05:49:14.259776 9789 instancegroups.go:187] Rolling updates for InstanceGroup node-2 are disabled I0920 05:49:14.259792 9789 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabled (0.00s) === RUN TestRollingUpdateDisabledSurge I0920 05:49:14.260045 9789 instancegroups.go:572] Detaching instance "bastion-1a", in group "bastion-1". I0920 05:49:14.260105 9789 instancegroups.go:174] waiting for 1ms after detaching instance I0920 05:49:14.261267 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.261317 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.266603 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.266641 9789 instancegroups.go:187] Rolling updates for InstanceGroup bastion-1 are disabled I0920 05:49:14.266708 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.266733 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.266747 9789 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I0920 05:49:14.267531 9789 instancegroups.go:187] Rolling updates for InstanceGroup master-1 are disabled I0920 05:49:14.267563 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.267582 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.267595 9789 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I0920 05:49:14.268530 9789 instancegroups.go:570] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0920 05:49:14.268590 9789 instancegroups.go:174] waiting for 1ms after detaching instance I0920 05:49:14.269762 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.269805 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.275079 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.275118 9789 instancegroups.go:187] Rolling updates for InstanceGroup node-1 are disabled I0920 05:49:14.275134 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.275153 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.275167 9789 instancegroups.go:313] Tainting 3 nodes in "node-2" instancegroup. I0920 05:49:14.276138 9789 instancegroups.go:570] Detaching instance "node-2c", node "node-2c.local", in group "node-2". I0920 05:49:14.276202 9789 instancegroups.go:174] waiting for 1ms after detaching instance I0920 05:49:14.277360 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.277405 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:14.282707 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.282757 9789 instancegroups.go:187] Rolling updates for InstanceGroup node-2 are disabled I0920 05:49:14.282780 9789 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabledSurge (0.02s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdate I0920 05:49:14.283116 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.283159 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.283178 9789 instancegroups.go:313] Tainting 7 nodes in "node-1" instancegroup. I0920 05:49:14.285647 9789 instancegroups.go:402] Draining the node: "node-1a.local". I0920 05:49:14.286418 9789 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0920 05:49:14.286465 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.287664 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.287710 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.287752 9789 instancegroups.go:402] Draining the node: "node-1c.local". I0920 05:49:14.287847 9789 instancegroups.go:402] Draining the node: "node-1b.local". I0920 05:49:14.289048 9789 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0920 05:49:14.289357 9789 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0920 05:49:14.289401 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.290591 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.290659 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.391211 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.391298 9789 instancegroups.go:402] Draining the node: "node-1e.local". I0920 05:49:14.391368 9789 instancegroups.go:402] Draining the node: "node-1d.local". I0920 05:49:14.392862 9789 instancegroups.go:593] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0920 05:49:14.393170 9789 instancegroups.go:593] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0920 05:49:14.393215 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.394426 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.394471 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.394516 9789 instancegroups.go:402] Draining the node: "node-1f.local". I0920 05:49:14.395541 9789 instancegroups.go:593] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0920 05:49:14.395588 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.396772 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.396834 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.497335 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.497390 9789 instancegroups.go:402] Draining the node: "node-1g.local". I0920 05:49:14.498404 9789 instancegroups.go:593] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0920 05:49:14.498451 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.499621 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.499678 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.499707 9789 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxUnavailableAllButOneNeedUpdate I0920 05:49:14.500145 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.500233 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.500261 9789 instancegroups.go:313] Tainting 6 nodes in "node-1" instancegroup. I0920 05:49:14.502668 9789 instancegroups.go:402] Draining the node: "node-1b.local". I0920 05:49:14.502761 9789 instancegroups.go:402] Draining the node: "node-1a.local". I0920 05:49:14.504175 9789 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0920 05:49:14.504525 9789 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0920 05:49:14.504587 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.505807 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.505878 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.606430 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.606502 9789 instancegroups.go:402] Draining the node: "node-1d.local". I0920 05:49:14.606575 9789 instancegroups.go:402] Draining the node: "node-1c.local". I0920 05:49:14.608016 9789 instancegroups.go:593] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0920 05:49:14.608398 9789 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0920 05:49:14.608460 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.609702 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.609773 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.609827 9789 instancegroups.go:402] Draining the node: "node-1e.local". I0920 05:49:14.610900 9789 instancegroups.go:593] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0920 05:49:14.610953 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.612146 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.612240 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.712695 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.712787 9789 instancegroups.go:402] Draining the node: "node-1f.local". I0920 05:49:14.714017 9789 instancegroups.go:593] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0920 05:49:14.714082 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.715279 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.715348 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.715381 9789 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllButOneNeedUpdate (0.22s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdateMaster I0920 05:49:14.715780 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.715832 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.715860 9789 instancegroups.go:313] Tainting 7 nodes in "master-1" instancegroup. I0920 05:49:14.719972 9789 instancegroups.go:402] Draining the node: "master-1a.local". I0920 05:49:14.721299 9789 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0920 05:49:14.721369 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.722572 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.722621 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.722669 9789 instancegroups.go:402] Draining the node: "master-1c.local". I0920 05:49:14.722741 9789 instancegroups.go:402] Draining the node: "master-1b.local". I0920 05:49:14.724056 9789 instancegroups.go:593] Stopping instance "master-1c", node "master-1c.local", in group "master-1" (this may take a while). I0920 05:49:14.724347 9789 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0920 05:49:14.724392 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.725571 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.725654 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.826049 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.826128 9789 instancegroups.go:402] Draining the node: "master-1e.local". I0920 05:49:14.826189 9789 instancegroups.go:402] Draining the node: "master-1d.local". I0920 05:49:14.827619 9789 instancegroups.go:593] Stopping instance "master-1e", node "master-1e.local", in group "master-1" (this may take a while). I0920 05:49:14.827911 9789 instancegroups.go:593] Stopping instance "master-1d", node "master-1d.local", in group "master-1" (this may take a while). I0920 05:49:14.827959 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.829190 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.829257 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.829313 9789 instancegroups.go:402] Draining the node: "master-1f.local". I0920 05:49:14.830368 9789 instancegroups.go:593] Stopping instance "master-1f", node "master-1f.local", in group "master-1" (this may take a while). I0920 05:49:14.830418 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.831631 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.831705 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.932353 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.932430 9789 instancegroups.go:402] Draining the node: "master-1g.local". I0920 05:49:14.933441 9789 instancegroups.go:593] Stopping instance "master-1g", node "master-1g.local", in group "master-1" (this may take a while). I0920 05:49:14.933494 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.934698 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.934744 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.934763 9789 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdateMaster (0.22s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdate I0920 05:49:14.935045 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.935080 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.935096 9789 instancegroups.go:313] Tainting 6 nodes in "node-1" instancegroup. I0920 05:49:14.937026 9789 instancegroups.go:570] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0920 05:49:14.937096 9789 instancegroups.go:174] waiting for 1ms after detaching instance I0920 05:49:14.938301 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.938345 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.938359 9789 instancegroups.go:570] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0920 05:49:14.938416 9789 instancegroups.go:174] waiting for 1ms after detaching instance I0920 05:49:14.939613 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.939656 9789 instancegroups.go:505] Cluster validated. I0920 05:49:14.939701 9789 instancegroups.go:402] Draining the node: "node-1b.local". I0920 05:49:14.939795 9789 instancegroups.go:402] Draining the node: "node-1a.local". I0920 05:49:14.941062 9789 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0920 05:49:14.941541 9789 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0920 05:49:14.941595 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:14.942823 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:14.942896 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.043523 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.043619 9789 instancegroups.go:402] Draining the node: "node-1d.local". I0920 05:49:15.043678 9789 instancegroups.go:402] Draining the node: "node-1c.local". I0920 05:49:15.045120 9789 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0920 05:49:15.045482 9789 instancegroups.go:593] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0920 05:49:15.045533 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.046726 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.046770 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.046813 9789 instancegroups.go:402] Draining the node: "node-1e.local". I0920 05:49:15.047764 9789 instancegroups.go:593] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0920 05:49:15.047815 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.049001 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.049074 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.149689 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.149747 9789 instancegroups.go:402] Draining the node: "node-1f.local". I0920 05:49:15.150790 9789 instancegroups.go:593] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0920 05:49:15.150841 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.152049 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.152092 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.152110 9789 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeAllButOneNeedUpdate I0920 05:49:15.152370 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.152404 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.152418 9789 instancegroups.go:313] Tainting 6 nodes in "node-1" instancegroup. I0920 05:49:15.154995 9789 instancegroups.go:570] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0920 05:49:15.155806 9789 instancegroups.go:570] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0920 05:49:15.155928 9789 instancegroups.go:174] waiting for 1ms after detaching instance I0920 05:49:15.157244 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.157302 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.157362 9789 instancegroups.go:402] Draining the node: "node-1b.local". I0920 05:49:15.157496 9789 instancegroups.go:402] Draining the node: "node-1a.local". I0920 05:49:15.158557 9789 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0920 05:49:15.158762 9789 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0920 05:49:15.158794 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.159929 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.160078 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.260598 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.260693 9789 instancegroups.go:402] Draining the node: "node-1d.local". I0920 05:49:15.260871 9789 instancegroups.go:402] Draining the node: "node-1c.local". I0920 05:49:15.262464 9789 instancegroups.go:593] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0920 05:49:15.262870 9789 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0920 05:49:15.262926 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.264122 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.264168 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.264209 9789 instancegroups.go:402] Draining the node: "node-1e.local". I0920 05:49:15.265202 9789 instancegroups.go:593] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0920 05:49:15.265252 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.266459 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.266530 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.367303 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.367369 9789 instancegroups.go:402] Draining the node: "node-1f.local". I0920 05:49:15.368401 9789 instancegroups.go:593] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0920 05:49:15.368454 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.369653 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.369731 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.369770 9789 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllButOneNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeGreaterThanNeedUpdate I0920 05:49:15.370213 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.370262 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.370288 9789 instancegroups.go:313] Tainting 2 nodes in "node-1" instancegroup. I0920 05:49:15.371757 9789 instancegroups.go:570] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0920 05:49:15.371814 9789 instancegroups.go:570] Detaching instance "node-1a", node "node-1a.local", in group "node-1". I0920 05:49:15.371883 9789 instancegroups.go:174] waiting for 1ms after detaching instance I0920 05:49:15.373107 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.373164 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:15.378418 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.378479 9789 instancegroups.go:402] Draining the node: "node-1b.local". I0920 05:49:15.378506 9789 instancegroups.go:402] Draining the node: "node-1a.local". I0920 05:49:15.379931 9789 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0920 05:49:15.380003 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.380285 9789 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0920 05:49:15.380327 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.381241 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.381275 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:15.386611 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.386662 9789 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeGreaterThanNeedUpdate (0.02s) === RUN TestRollingUpdateDetachFails I0920 05:49:15.386986 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.387014 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.387028 9789 instancegroups.go:313] Tainting 2 nodes in "node-1" instancegroup. I0920 05:49:15.387832 9789 instancegroups.go:570] Detaching instance "node-1b", node "node-1b.local", in group "node-1". E0920 05:49:15.387898 9789 instancegroups.go:162] Failed to detach instance "node-1b": error detaching instance "node-1b", node "node-1b.local": error detaching instance "node-1b": testing error I0920 05:49:15.387912 9789 instancegroups.go:570] Detaching instance "node-1a", node "node-1a.local", in group "node-1". E0920 05:49:15.387970 9789 instancegroups.go:162] Failed to detach instance "node-1a": error detaching instance "node-1a", node "node-1a.local": error detaching instance "node-1a": testing error I0920 05:49:15.388004 9789 instancegroups.go:174] waiting for 1ms after detaching instance I0920 05:49:15.389193 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.389227 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:15.394541 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.394597 9789 instancegroups.go:402] Draining the node: "node-1b.local". I0920 05:49:15.394617 9789 instancegroups.go:402] Draining the node: "node-1a.local". I0920 05:49:15.396140 9789 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0920 05:49:15.396180 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.396474 9789 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0920 05:49:15.396509 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.397811 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.397848 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:15.403125 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.403161 9789 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDetachFails (0.02s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached I0920 05:49:15.403410 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.403445 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.403459 9789 instancegroups.go:313] Tainting 4 nodes in "node-1" instancegroup. I0920 05:49:15.404757 9789 instancegroups.go:570] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0920 05:49:15.404818 9789 instancegroups.go:174] waiting for 1ms after detaching instance I0920 05:49:15.406036 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.406087 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.406101 9789 instancegroups.go:570] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0920 05:49:15.406154 9789 instancegroups.go:174] waiting for 1ms after detaching instance I0920 05:49:15.407364 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.407408 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.407455 9789 instancegroups.go:402] Draining the node: "node-1c.local". I0920 05:49:15.407528 9789 instancegroups.go:402] Draining the node: "node-1a.local". I0920 05:49:15.407595 9789 instancegroups.go:402] Draining the node: "node-1b.local". I0920 05:49:15.409346 9789 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0920 05:49:15.409406 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.409699 9789 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0920 05:49:15.409752 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.410034 9789 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0920 05:49:15.410078 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.410637 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.511577 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.511638 9789 instancegroups.go:402] Draining the node: "node-1d.local". I0920 05:49:15.512569 9789 instancegroups.go:593] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0920 05:49:15.512621 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.513849 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.513926 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.513949 9789 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached (0.11s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached I0920 05:49:15.514254 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.514289 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.514305 9789 instancegroups.go:313] Tainting 7 nodes in "node-1" instancegroup. I0920 05:49:15.516546 9789 instancegroups.go:402] Draining the node: "node-1a.local". I0920 05:49:15.517694 9789 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0920 05:49:15.517762 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.518988 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.519034 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.519088 9789 instancegroups.go:402] Draining the node: "node-1e.local". I0920 05:49:15.519115 9789 instancegroups.go:402] Draining the node: "node-1c.local". I0920 05:49:15.520526 9789 instancegroups.go:593] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0920 05:49:15.520876 9789 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0920 05:49:15.520930 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.522116 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.522213 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.623102 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.623176 9789 instancegroups.go:402] Draining the node: "node-1g.local". I0920 05:49:15.623262 9789 instancegroups.go:402] Draining the node: "node-1f.local". I0920 05:49:15.624943 9789 instancegroups.go:593] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0920 05:49:15.625291 9789 instancegroups.go:593] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0920 05:49:15.625337 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.626555 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.626604 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.626665 9789 instancegroups.go:402] Draining the node: "node-1b.local". I0920 05:49:15.627696 9789 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0920 05:49:15.627747 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.628959 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.629031 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.729937 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.730029 9789 instancegroups.go:402] Draining the node: "node-1d.local". I0920 05:49:15.731118 9789 instancegroups.go:593] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0920 05:49:15.731174 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.732389 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.732435 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.732455 9789 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached (0.22s) === RUN TestRollingUpdateOnlyWarmPoolNodes I0920 05:49:15.732763 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.732794 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.732809 9789 instancegroups.go:112] deleting warm pool instance "node-1-wp-a" I0920 05:49:15.732829 9789 instancegroups.go:112] deleting warm pool instance "node-1-wp-b" I0920 05:49:15.732853 9789 instancegroups.go:112] deleting warm pool instance "node-1-wp-c" I0920 05:49:15.732883 9789 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateOnlyWarmPoolNodes (0.00s) === RUN TestRollingWarmPoolBeforeJoinedNodes I0920 05:49:15.733132 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.733166 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.733201 9789 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I0920 05:49:15.734341 9789 instancegroups.go:112] deleting warm pool instance "node-1-wp-a" I0920 05:49:15.734378 9789 instancegroups.go:112] deleting warm pool instance "node-1-wp-b" I0920 05:49:15.734395 9789 instancegroups.go:112] deleting warm pool instance "node-1-wp-c" I0920 05:49:15.734447 9789 instancegroups.go:402] Draining the node: "node-1a.local". I0920 05:49:15.735282 9789 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0920 05:49:15.735321 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.736526 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.736562 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:15.741796 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.741848 9789 instancegroups.go:402] Draining the node: "node-1b.local". I0920 05:49:15.742729 9789 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0920 05:49:15.742771 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.743966 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.744002 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:15.749249 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.749300 9789 instancegroups.go:402] Draining the node: "node-1c.local". I0920 05:49:15.750251 9789 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0920 05:49:15.750293 9789 instancegroups.go:438] waiting for 1ms after terminating instance I0920 05:49:15.751497 9789 instancegroups.go:472] Validating the cluster. I0920 05:49:15.751532 9789 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I0920 05:49:15.756803 9789 instancegroups.go:505] Cluster validated. I0920 05:49:15.756872 9789 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingWarmPoolBeforeJoinedNodes (0.02s) === RUN TestSettings === RUN TestSettings/DrainAndTerminate === RUN TestSettings/MaxUnavailable === RUN TestSettings/MaxSurge --- PASS: TestSettings (0.00s) --- PASS: TestSettings/DrainAndTerminate (0.00s) --- PASS: TestSettings/MaxUnavailable (0.00s) --- PASS: TestSettings/MaxSurge (0.00s) === RUN TestMaxUnavailable === RUN TestMaxUnavailable/0_1 === RUN TestMaxUnavailable/0%_1 === RUN TestMaxUnavailable/39%_10 === RUN TestMaxUnavailable/100%_10 --- PASS: TestMaxUnavailable (0.00s) --- PASS: TestMaxUnavailable/0_1 (0.00s) --- PASS: TestMaxUnavailable/0%_1 (0.00s) --- PASS: TestMaxUnavailable/39%_10 (0.00s) --- PASS: TestMaxUnavailable/100%_10 (0.00s) === RUN TestMaxSurge === RUN TestMaxSurge/0_1 === RUN TestMaxSurge/0%_1 === RUN TestMaxSurge/31%_10 === RUN TestMaxSurge/100%_10 --- PASS: TestMaxSurge (0.00s) --- PASS: TestMaxSurge/0_1 (0.00s) --- PASS: TestMaxSurge/0%_1 (0.00s) --- PASS: TestMaxSurge/31%_10 (0.00s) --- PASS: TestMaxSurge/100%_10 (0.00s) === RUN TestAWSDefault --- PASS: TestAWSDefault (0.00s) PASS ok k8s.io/kops/pkg/instancegroups 2.018s === RUN TestWriteToken === RUN TestWriteToken/WriteJson === RUN TestWriteToken/WriteJsonWithIndent --- PASS: TestWriteToken (0.00s) --- PASS: TestWriteToken/WriteJson (0.00s) --- PASS: TestWriteToken/WriteJsonWithIndent (0.00s) PASS ok k8s.io/kops/pkg/jsonutils 0.109s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.00s) PASS ok k8s.io/kops/pkg/k8scodecs 0.069s === RUN TestParse E0920 05:48:48.860229 5585 versions.go:96] unable to parse Kubernetes version "" E0920 05:48:48.860300 5585 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: TestParse (0.00s) === RUN TestIsGTE === RUN TestIsGTE/KV_greater_than_Version === RUN TestIsGTE/KV_greater_than_Version#01 === RUN TestIsGTE/KV_equal_Version === RUN TestIsGTE/KV_equal_Version#01 === RUN TestIsGTE/Version_greater_than_KV === RUN TestIsGTE/Version_greater_than_KV#01 --- PASS: TestIsGTE (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version#01 (0.00s) --- PASS: TestIsGTE/KV_equal_Version (0.00s) --- PASS: TestIsGTE/KV_equal_Version#01 (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV#01 (0.00s) PASS ok k8s.io/kops/pkg/k8sversion 0.037s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.01s) === RUN TestToVersionedJSON --- PASS: TestToVersionedJSON (0.00s) === RUN TestRewriteAPIGroup --- PASS: TestRewriteAPIGroup (0.00s) PASS ok k8s.io/kops/pkg/kopscodecs 0.069s === RUN TestBuildKubecfg === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster === RUN TestBuildKubecfg/Public_DNS_with_kops_auth_plugin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port --- PASS: TestBuildKubecfg (0.22s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin (0.05s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port (0.04s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate (0.04s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate (0.01s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster (0.00s) --- PASS: TestBuildKubecfg/Public_DNS_with_kops_auth_plugin (0.01s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin (0.04s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port (0.02s) PASS ok k8s.io/kops/pkg/kubeconfig 0.354s ? k8s.io/kops/pkg/kubemanifest [no test files] === RUN Test_ProxyFunc --- PASS: Test_ProxyFunc (0.00s) === RUN TestBootstrapUserData --- PASS: TestBootstrapUserData (0.01s) === RUN TestValidateAWSVolumeAllow50ratio --- PASS: TestValidateAWSVolumeAllow50ratio (0.00s) PASS ok k8s.io/kops/pkg/model 0.083s === RUN TestRootVolumeOptimizationFlag --- PASS: TestRootVolumeOptimizationFlag (0.00s) === RUN TestAPIServerAdditionalSecurityGroupsWithNLB === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Master === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Node --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Master (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Node (0.00s) === RUN TestJoinSuffixes --- PASS: TestJoinSuffixes (0.00s) === RUN TestIAMServiceEC2 --- PASS: TestIAMServiceEC2 (0.00s) === RUN Test_formatAWSIAMStatement === RUN Test_formatAWSIAMStatement/namespace_and_name_without_wildcard === RUN Test_formatAWSIAMStatement/name_contains_wildcard === RUN Test_formatAWSIAMStatement/namespace_contains_wildcard --- PASS: Test_formatAWSIAMStatement (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_and_name_without_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/name_contains_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_contains_wildcard (0.00s) PASS ok k8s.io/kops/pkg/model/awsmodel 0.052s === RUN TestAPILoadBalancerModelBuilder_Build --- PASS: TestAPILoadBalancerModelBuilder_Build (0.00s) === RUN TestSubnetForLoadbalancer --- PASS: TestSubnetForLoadbalancer (0.00s) === RUN TestCloudTagsForInstanceGroup --- PASS: TestCloudTagsForInstanceGroup (0.00s) === RUN TestNetworkModelBuilder_Build --- PASS: TestNetworkModelBuilder_Build (0.00s) === RUN TestResourceGroupModelBuilder_Build --- PASS: TestResourceGroupModelBuilder_Build (0.00s) === RUN TestVMScaleSetModelBuilder_Build --- PASS: TestVMScaleSetModelBuilder_Build (0.00s) === RUN TestGetCapacity === RUN TestGetCapacity/test_case_0 === RUN TestGetCapacity/test_case_1 === RUN TestGetCapacity/test_case_2 === RUN TestGetCapacity/test_case_3 --- PASS: TestGetCapacity (0.00s) --- PASS: TestGetCapacity/test_case_0 (0.00s) --- PASS: TestGetCapacity/test_case_1 (0.00s) --- PASS: TestGetCapacity/test_case_2 (0.00s) --- PASS: TestGetCapacity/test_case_3 (0.00s) === RUN TestGetStorageProfile === RUN TestGetStorageProfile/test_case_0 === RUN TestGetStorageProfile/test_case_1 --- PASS: TestGetStorageProfile (0.00s) --- PASS: TestGetStorageProfile/test_case_0 (0.00s) --- PASS: TestGetStorageProfile/test_case_1 (0.00s) === RUN TestParseImage === RUN TestParseImage/test_case_0 === RUN TestParseImage/test_case_1 === RUN TestParseImage/test_case_2 === RUN TestParseImage/test_case_3 --- PASS: TestParseImage (0.00s) --- PASS: TestParseImage/test_case_0 (0.00s) --- PASS: TestParseImage/test_case_1 (0.00s) --- PASS: TestParseImage/test_case_2 (0.00s) --- PASS: TestParseImage/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/model/azuremodel 0.068s === RUN TestCloudConfigurationOptionsBuilder === RUN TestCloudConfigurationOptionsBuilder/neither === RUN TestCloudConfigurationOptionsBuilder/all_false === RUN TestCloudConfigurationOptionsBuilder/all_true === RUN TestCloudConfigurationOptionsBuilder/os_false === RUN TestCloudConfigurationOptionsBuilder/os_true === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_true === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_true --- PASS: TestCloudConfigurationOptionsBuilder (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/neither (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_true (0.00s) === RUN Test_Build_Containerd_Supported_Version --- PASS: Test_Build_Containerd_Supported_Version (0.00s) === RUN Test_Build_Containerd_Unneeded_Runtime --- PASS: Test_Build_Containerd_Unneeded_Runtime (0.00s) === RUN Test_Build_Containerd_Needed_Runtime --- PASS: Test_Build_Containerd_Needed_Runtime (0.00s) === RUN TestImage I0920 05:48:57.366270 7824 featureflag.go:160] FeatureFlag "ImageDigest"=false --- PASS: TestImage (0.00s) === RUN Test_Build_KCM_Builder --- PASS: Test_Build_KCM_Builder (0.00s) === RUN Test_Build_KCM_Builder_Change_Duration --- PASS: Test_Build_KCM_Builder_Change_Duration (0.00s) === RUN Test_Build_KCM_Builder_CIDR_Mask_Size === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: (0.00s) === RUN TestFeatureGatesKubernetesVersion --- PASS: TestFeatureGatesKubernetesVersion (0.00s) === RUN TestFeatureGatesOverride --- PASS: TestFeatureGatesOverride (0.00s) === RUN Test_Build_Scheduler_Without_PolicyConfigMap --- PASS: Test_Build_Scheduler_Without_PolicyConfigMap (0.00s) === RUN Test_Build_Scheduler_PolicyConfigMap_Supported_Version --- PASS: Test_Build_Scheduler_PolicyConfigMap_Supported_Version (0.00s) PASS ok k8s.io/kops/pkg/model/components 0.186s ? k8s.io/kops/pkg/model/components/addonmanifests [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awscloudcontrollermanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsebscsidriver [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsloadbalancercontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/certmanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/clusterautoscaler [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/dnscontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/externaldns [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/karpenter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/kuberouter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/nodeterminationhandler [no test files] === RUN Test_RunEtcdManagerBuilder I0920 05:49:00.860841 8978 featureflag.go:160] FeatureFlag "ImageDigest"=false === RUN Test_RunEtcdManagerBuilder/basedir=tests/minimal === RUN Test_RunEtcdManagerBuilder/basedir=tests/interval === RUN Test_RunEtcdManagerBuilder/basedir=tests/proxy === RUN Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings W0920 05:49:00.888995 8978 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 3 W0920 05:49:00.890796 8978 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 3 --- PASS: Test_RunEtcdManagerBuilder (0.03s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/minimal (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/interval (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/proxy (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings (0.01s) PASS ok k8s.io/kops/pkg/model/components/etcdmanager 0.106s ? k8s.io/kops/pkg/model/components/kopscontroller [no test files] === RUN Test_RunKubeApiserverBuilder I0920 05:49:00.930740 8996 featureflag.go:160] FeatureFlag "ImageDigest"=false === RUN Test_RunKubeApiserverBuilder/basedir=tests/minimal --- PASS: Test_RunKubeApiserverBuilder (0.01s) --- PASS: Test_RunKubeApiserverBuilder/basedir=tests/minimal (0.01s) PASS ok k8s.io/kops/pkg/model/components/kubeapiserver 0.085s === RUN Test_RunKubeSchedulerBuilder === RUN Test_RunKubeSchedulerBuilder/basedir=tests/minimal === RUN Test_RunKubeSchedulerBuilder/basedir=tests/kubeschedulerconfig === RUN Test_RunKubeSchedulerBuilder/basedir=tests/mixing --- PASS: Test_RunKubeSchedulerBuilder (0.03s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/minimal (0.03s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/kubeschedulerconfig (0.00s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/mixing (0.00s) PASS ok k8s.io/kops/pkg/model/components/kubescheduler 0.189s === RUN TestDefaultInstanceGroupVolumeSize --- PASS: TestDefaultInstanceGroupVolumeSize (0.00s) PASS ok k8s.io/kops/pkg/model/defaults 0.045s ? k8s.io/kops/pkg/model/domodel [no test files] ? k8s.io/kops/pkg/model/gcemodel [no test files] ? k8s.io/kops/pkg/model/hetznermodel [no test files] === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) === RUN TestPolicyGeneration --- PASS: TestPolicyGeneration (0.00s) === RUN TestEmptyPolicy --- PASS: TestEmptyPolicy (0.00s) PASS ok k8s.io/kops/pkg/model/iam 0.101s === RUN TestServerGroupBuilder I0920 05:49:02.025388 9303 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.035952 9303 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.048211 9303 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.064486 9303 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.114637 9303 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.130280 9303 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.150868 9303 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.163945 9303 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.170757 9303 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.226412 9303 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.235301 9303 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.242709 9303 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.250038 9303 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.255455 9303 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:02.263723 9303 featureflag.go:160] FeatureFlag "ImageDigest"=false --- PASS: TestServerGroupBuilder (0.25s) PASS ok k8s.io/kops/pkg/model/openstackmodel 0.304s === RUN Test_NodeUpTabs --- PASS: Test_NodeUpTabs (0.00s) PASS ok k8s.io/kops/pkg/model/resources 0.045s ? k8s.io/kops/pkg/nodeidentity [no test files] ? k8s.io/kops/pkg/nodeidentity/aws [no test files] === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetVMSSNameFromProviderID === RUN TestGetVMSSNameFromProviderID/test_case_0 === RUN TestGetVMSSNameFromProviderID/test_case_1 === RUN TestGetVMSSNameFromProviderID/test_case_2 --- PASS: TestGetVMSSNameFromProviderID (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_0 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_1 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_2 (0.00s) === RUN TestIdentifyNode === RUN TestIdentifyNode/test_case_0 === RUN TestIdentifyNode/test_case_1 --- PASS: TestIdentifyNode (0.00s) --- PASS: TestIdentifyNode/test_case_0 (0.00s) --- PASS: TestIdentifyNode/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/nodeidentity/azure 0.047s ? k8s.io/kops/pkg/nodeidentity/do [no test files] ? k8s.io/kops/pkg/nodeidentity/gce [no test files] ? k8s.io/kops/pkg/nodeidentity/hetzner [no test files] ? k8s.io/kops/pkg/nodeidentity/openstack [no test files] === RUN TestBuildNodeLabels === RUN TestBuildNodeLabels/RoleControlPlane === RUN TestBuildNodeLabels/RoleNode --- PASS: TestBuildNodeLabels (0.00s) --- PASS: TestBuildNodeLabels/RoleControlPlane (0.00s) --- PASS: TestBuildNodeLabels/RoleNode (0.00s) PASS ok k8s.io/kops/pkg/nodelabels 0.119s === RUN TestGenerateCertificate === RUN TestGenerateCertificate/selfsigned === RUN TestGenerateCertificate/client === RUN TestGenerateCertificate/server --- PASS: TestGenerateCertificate (0.01s) --- PASS: TestGenerateCertificate/selfsigned (0.01s) --- PASS: TestGenerateCertificate/client (0.00s) --- PASS: TestGenerateCertificate/server (0.00s) === RUN TestCertificateRoundTrip --- PASS: TestCertificateRoundTrip (0.00s) === RUN TestIssueCert === RUN TestIssueCert/ca === RUN TestIssueCert/client === RUN TestIssueCert/clientOneYear === RUN TestIssueCert/#00 === RUN TestIssueCert/server --- PASS: TestIssueCert (0.97s) --- PASS: TestIssueCert/ca (0.23s) --- PASS: TestIssueCert/client (0.59s) --- PASS: TestIssueCert/clientOneYear (0.15s) --- PASS: TestIssueCert/#00 (0.00s) --- PASS: TestIssueCert/server (0.00s) === RUN TestPrivateKeyRoundTripRSA === RUN TestPrivateKeyRoundTripRSA/rsa === RUN TestPrivateKeyRoundTripRSA/ecdsa --- PASS: TestPrivateKeyRoundTripRSA (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/rsa (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/ecdsa (0.00s) === RUN Test_AWSFingerprint_RsaKey1 --- PASS: Test_AWSFingerprint_RsaKey1 (0.00s) === RUN Test_AWSFingerprint_RsaKeyEncrypted --- PASS: Test_AWSFingerprint_RsaKeyEncrypted (0.00s) === RUN Test_AWSFingerprint_Ed25519Key --- PASS: Test_AWSFingerprint_Ed25519Key (0.00s) === RUN Test_AWSFingerprint_TrickyWhitespace --- PASS: Test_AWSFingerprint_TrickyWhitespace (0.00s) === RUN Test_AWSFingerprint_DsaKey --- PASS: Test_AWSFingerprint_DsaKey (0.00s) === RUN Test_OpenSSHFingerprint_RsaKey1 --- PASS: Test_OpenSSHFingerprint_RsaKey1 (0.00s) PASS ok k8s.io/kops/pkg/pki 0.996s === RUN TestLongDesc --- PASS: TestLongDesc (0.00s) PASS ok k8s.io/kops/pkg/pretty 0.043s ? k8s.io/kops/pkg/rbac [no test files] ? k8s.io/kops/pkg/resolver [no test files] ? k8s.io/kops/pkg/resources [no test files] === RUN TestAddUntaggedRouteTables I0920 05:49:01.707711 9169 routetable.go:53] DescribeRouteTables: { } W0920 05:49:01.707865 9169 tags.go:67] cluster tag not found on route-table:rtb-1234 I0920 05:49:01.707880 9169 aws.go:226] Skipping route table in VPC, but with wrong cluster tag ("other.example.com") --- PASS: TestAddUntaggedRouteTables (0.00s) === RUN TestListIAMInstanceProfiles I0920 05:49:01.708019 9169 iaminstanceprofile.go:193] ListInstanceProfiles: { } W0920 05:49:01.708097 9169 aws.go:2097] could not find instance profile "__no_entity__.me.example.com". Resource may already have been deleted: NoSuchEntity: No such entity --- PASS: TestListIAMInstanceProfiles (0.00s) === RUN TestListIAMRoles I0920 05:49:01.708186 9169 iamrole.go:89] ListRoles: { } --- PASS: TestListIAMRoles (0.00s) === RUN TestListRouteTables I0920 05:49:01.708275 9169 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["me.example.com"] }] } I0920 05:49:01.708348 9169 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/me.example.com"] }] } --- PASS: TestListRouteTables (0.00s) === RUN TestSharedVolume I0920 05:49:01.708420 9169 volumes.go:33] CreateVolume: { TagSpecifications: [{ ResourceType: "volume", Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "shared" }] }] } I0920 05:49:01.708512 9169 volumes.go:33] CreateVolume: { TagSpecifications: [{ Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "owned" }] }] } I0920 05:49:01.708547 9169 volumes.go:128] DescribeVolumes: { } W0920 05:49:01.708579 9169 aws.go:2250] (new) cluster tag not found on volume:vol-2 aws_test.go:334: 2 --- PASS: TestSharedVolume (0.00s) === RUN TestMatchesElbTags --- PASS: TestMatchesElbTags (0.00s) === RUN TestListVPCs I0920 05:49:01.708716 9169 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:01.708888 9169 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:01.708927 9169 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:01.708994 9169 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:01.709053 9169 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:01.709099 9169 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:01.709122 9169 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:01.709156 9169 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:01.709183 9169 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:01.709210 9169 tags.go:42] CreateTags { Resources: ["vpc-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "legacy.example.com" }] } I0920 05:49:01.709248 9169 tags.go:42] CreateTags { Resources: ["vpc-shared"], Tags: [{ Key: "kubernetes.io/cluster/shared.example.com", Value: "shared" }] } I0920 05:49:01.709277 9169 tags.go:42] CreateTags { Resources: ["vpc-shared-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "shared-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/shared-with-legacy.example.com", Value: "shared" }] } I0920 05:49:01.709317 9169 tags.go:42] CreateTags { Resources: ["vpc-owned"], Tags: [{ Key: "kubernetes.io/cluster/owned.example.com", Value: "owned" }] } I0920 05:49:01.709345 9169 tags.go:42] CreateTags { Resources: ["vpc-owned-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "owned-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/owned-with-legacy.example.com", Value: "owned" }] } I0920 05:49:01.709379 9169 tags.go:42] CreateTags { Resources: ["vpc-other"], Tags: [{ Key: "KubernetesCluster", Value: "other.example.com" },{ Key: "kubernetes.io/cluster/other.example.com", Value: "shared" }] } I0920 05:49:01.709417 9169 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["mismatch.example.com"] }] } I0920 05:49:01.709456 9169 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/mismatch.example.com"] }] } I0920 05:49:01.709483 9169 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["legacy.example.com"] }] } I0920 05:49:01.709510 9169 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/legacy.example.com"] }] } I0920 05:49:01.709538 9169 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared-with-legacy.example.com"] }] } I0920 05:49:01.709562 9169 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared-with-legacy.example.com"] }] } I0920 05:49:01.709595 9169 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared.example.com"] }] } I0920 05:49:01.709618 9169 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared.example.com"] }] } I0920 05:49:01.709656 9169 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned-with-legacy.example.com"] }] } I0920 05:49:01.709682 9169 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned-with-legacy.example.com"] }] } I0920 05:49:01.709709 9169 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned.example.com"] }] } I0920 05:49:01.709732 9169 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned.example.com"] }] } --- PASS: TestListVPCs (0.00s) PASS ok k8s.io/kops/pkg/resources/aws 0.071s === RUN TestListResourcesAzure --- PASS: TestListResourcesAzure (0.00s) === RUN TestIsOwnedByCluster === RUN TestIsOwnedByCluster/test_case_0 === RUN TestIsOwnedByCluster/test_case_1 === RUN TestIsOwnedByCluster/test_case_2 === RUN TestIsOwnedByCluster/test_case_3 --- PASS: TestIsOwnedByCluster (0.00s) --- PASS: TestIsOwnedByCluster/test_case_0 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_1 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_2 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/resources/azure 0.046s ? k8s.io/kops/pkg/resources/digitalocean [no test files] === RUN TestNameMatch --- PASS: TestNameMatch (0.00s) === RUN TestMatchesClusterNameWithUUID --- PASS: TestMatchesClusterNameWithUUID (0.00s) PASS ok k8s.io/kops/pkg/resources/gce 0.116s ? k8s.io/kops/pkg/resources/hetzner [no test files] ? k8s.io/kops/pkg/resources/openstack [no test files] ? k8s.io/kops/pkg/resources/ops [no test files] ? k8s.io/kops/pkg/resources/spotinst [no test files] ? k8s.io/kops/pkg/sshcredentials [no test files] === RUN TestEscapeCommand --- PASS: TestEscapeCommand (0.00s) === RUN TestRawManifest --- PASS: TestRawManifest (0.00s) === RUN TestRawMixedManifest --- PASS: TestRawMixedManifest (0.00s) === RUN TestKeyPairOnlyManifest --- PASS: TestKeyPairOnlyManifest (0.00s) === RUN TestUnitFileExtensionValid === RUN TestUnitFileExtensionValid/valid_extension:_.automount === RUN TestUnitFileExtensionValid/valid_extension:_.device === RUN TestUnitFileExtensionValid/valid_extension:_.mount === RUN TestUnitFileExtensionValid/valid_extension:_.path === RUN TestUnitFileExtensionValid/valid_extension:_.scope === RUN TestUnitFileExtensionValid/valid_extension:_.service === RUN TestUnitFileExtensionValid/valid_extension:_.slice === RUN TestUnitFileExtensionValid/valid_extension:_.socket === RUN TestUnitFileExtensionValid/valid_extension:_.swap === RUN TestUnitFileExtensionValid/valid_extension:_.target === RUN TestUnitFileExtensionValid/valid_extension:_.timer === RUN TestUnitFileExtensionValid/invalid_extension:_(no_extension) === RUN TestUnitFileExtensionValid/invalid_extension:_.not-valid --- PASS: TestUnitFileExtensionValid (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.automount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.device (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.mount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.path (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.scope (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.service (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.slice (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.socket (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.swap (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.target (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.timer (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_(no_extension) (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_.not-valid (0.00s) PASS ok k8s.io/kops/pkg/systemd 0.034s ? k8s.io/kops/pkg/templates [no test files] ? k8s.io/kops/pkg/testutils [no test files] ? k8s.io/kops/pkg/testutils/golden [no test files] ? k8s.io/kops/pkg/tokens [no test files] === RUN TestTruncateString === RUN TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false --- PASS: TestTruncateString (0.00s) --- PASS: TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false (0.00s) PASS ok k8s.io/kops/pkg/truncate 0.060s ? k8s.io/kops/pkg/try [no test files] === RUN TestJoin --- PASS: TestJoin (0.00s) PASS ok k8s.io/kops/pkg/urls 0.025s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) PASS ok k8s.io/kops/pkg/util/stringorslice 0.021s === RUN Test_Overlap --- PASS: Test_Overlap (0.00s) === RUN Test_BelongsTo --- PASS: Test_BelongsTo (0.00s) === RUN Test_SplitInto8 --- PASS: Test_SplitInto8 (0.00s) PASS ok k8s.io/kops/pkg/util/subnet 0.006s === RUN TestRenderGeneralOK --- PASS: TestRenderGeneralOK (0.01s) === RUN TestRenderMissingValue --- PASS: TestRenderMissingValue (0.00s) === RUN TestRenderIndent --- PASS: TestRenderIndent (0.00s) === RUN TestRenderChannelFunctions --- PASS: TestRenderChannelFunctions (0.03s) === RUN TestRenderSnippet --- PASS: TestRenderSnippet (0.00s) === RUN TestRenderContext --- PASS: TestRenderContext (0.00s) === RUN TestAllowForMissingVars --- PASS: TestAllowForMissingVars (0.00s) === RUN TestRenderIntegration --- PASS: TestRenderIntegration (0.00s) PASS ok k8s.io/kops/pkg/util/templater 0.151s === RUN Test_ValidateCloudGroupMissing --- PASS: Test_ValidateCloudGroupMissing (0.00s) === RUN Test_ValidateNodesNotEnough --- PASS: Test_ValidateNodesNotEnough (0.00s) === RUN Test_ValidateDetachedNodesDontCount --- PASS: Test_ValidateDetachedNodesDontCount (0.00s) === RUN Test_ValidateNodeNotReady --- PASS: Test_ValidateNodeNotReady (0.00s) === RUN Test_ValidateMastersNotEnough --- PASS: Test_ValidateMastersNotEnough (0.00s) === RUN Test_ValidateMasterNotReady --- PASS: Test_ValidateMasterNotReady (0.00s) === RUN Test_ValidateMasterStaticPods --- PASS: Test_ValidateMasterStaticPods (0.00s) === RUN Test_ValidateNoPodFailures --- PASS: Test_ValidateNoPodFailures (0.00s) === RUN Test_ValidatePodFailure === RUN Test_ValidatePodFailure/pending-node-kube-system === RUN Test_ValidatePodFailure/pending-node-kube-system#01 === RUN Test_ValidatePodFailure/pending-node-kube-system#02 === RUN Test_ValidatePodFailure/pending-node-otherNamespace === RUN Test_ValidatePodFailure/pending-node-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-node-otherNamespace#02 === RUN Test_ValidatePodFailure/pending-cluster-kube-system === RUN Test_ValidatePodFailure/pending-cluster-kube-system#01 === RUN Test_ValidatePodFailure/pending-cluster-kube-system#02 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-node-kube-system === RUN Test_ValidatePodFailure/notready-node-kube-system#01 === RUN Test_ValidatePodFailure/notready-node-kube-system#02 === RUN Test_ValidatePodFailure/notready-node-otherNamespace === RUN Test_ValidatePodFailure/notready-node-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-node-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-cluster-kube-system === RUN Test_ValidatePodFailure/notready-cluster-kube-system#01 === RUN Test_ValidatePodFailure/notready-cluster-kube-system#02 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-node-kube-system === RUN Test_ValidatePodFailure/unknown-node-kube-system#01 === RUN Test_ValidatePodFailure/unknown-node-kube-system#02 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#01 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#02 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 --- PASS: Test_ValidatePodFailure (0.01s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 (0.00s) === RUN Test_ValidateBastionNodes === RUN Test_ValidateBastionNodes/instancegroup's_nodes_not_ready === RUN Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready --- PASS: Test_ValidateBastionNodes (0.00s) --- PASS: Test_ValidateBastionNodes/instancegroup's_nodes_not_ready (0.00s) --- PASS: Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready (0.00s) === RUN Test_ValidateDetachedNodesNotValidated --- PASS: Test_ValidateDetachedNodesNotValidated (0.00s) PASS ok k8s.io/kops/pkg/validation 0.075s ? k8s.io/kops/pkg/values [no test files] ? k8s.io/kops/pkg/wellknownoperators [no test files] ? k8s.io/kops/pkg/wellknownports [no test files] ? k8s.io/kops/pkg/wellknownusers [no test files] ? k8s.io/kops/pkg/zones [no test files] ? k8s.io/kops/protokube/cmd/protokube [no test files] ? k8s.io/kops/protokube/pkg/etcd [no test files] ? k8s.io/kops/protokube/pkg/gossip [no test files] ? k8s.io/kops/protokube/pkg/gossip/aws [no test files] === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetInternalIP --- PASS: TestGetInternalIP (0.00s) === RUN TestGetSeeds --- PASS: TestGetSeeds (0.00s) PASS ok k8s.io/kops/protokube/pkg/gossip/azure 0.029s ? k8s.io/kops/protokube/pkg/gossip/dns [no test files] === RUN TestRemovesDuplicateGuardedBlocks W0920 05:48:51.707007 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.707111 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.707143 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.707171 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.707203 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.707232 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.707264 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.707294 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.707320 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.707347 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.707374 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.707401 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.707434 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.707462 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.707489 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.707516 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.716979 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.717082 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.717177 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.717213 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.717292 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.717326 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.717418 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.717453 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.717530 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.717564 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.717650 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.717688 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.717787 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.717821 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.717922 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.717958 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.718042 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.718077 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.718158 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.718193 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.718270 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.718304 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.718408 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.718443 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.718517 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.718549 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.718632 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.718667 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.718745 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.718779 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.718867 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.718883 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.718949 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.718966 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.719020 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.719034 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.719093 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.719125 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.735864 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.736134 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.736280 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.736421 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.736580 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.736734 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.736887 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.737006 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.737125 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.737254 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.737362 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.737480 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.737597 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.737716 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.737859 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.738007 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.738130 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.738253 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.738365 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.738475 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.738580 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.738707 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.738809 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.738933 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.739061 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.739183 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.739312 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.740324 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.742446 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.742476 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.742548 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.742577 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.742644 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.742659 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.742745 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.742764 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.742842 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.742860 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.742922 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.742944 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.743011 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.743026 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.743082 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.743097 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.743196 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.743212 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.743304 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.743334 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.743402 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.743418 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.743473 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.743487 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.743547 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.743562 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.743637 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.743652 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.743722 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.743737 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.743800 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.743814 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.745482 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.745509 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.745580 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.745594 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.745673 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.745693 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.745750 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.745763 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.745834 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.745854 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.745917 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.745931 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.746003 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.746017 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.746069 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.746083 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.746132 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.746146 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.746208 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.746222 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.746282 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.746295 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.746350 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.746363 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.746416 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.746430 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.746491 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.746505 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.746616 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.746631 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.746717 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.746733 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.746785 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.746799 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.746856 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.746872 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.748586 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.748754 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.748876 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.749022 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.749148 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.749308 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.749423 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.749535 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.749646 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.749766 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.749886 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.750018 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.750147 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.750288 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.750432 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.750559 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.750668 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.750787 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.750889 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.751049 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.751194 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.751332 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.751434 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.751535 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.751653 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.751766 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.751877 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.751984 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.752094 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.752202 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.752304 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.752410 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.752521 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.752622 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.752777 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.752962 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.753098 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.753243 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.753362 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.753470 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.753577 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.753690 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.753790 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.753890 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.753995 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.754109 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.754218 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.754324 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.754434 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.754540 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.754657 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.754775 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.754876 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.754982 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.755085 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.755186 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.755303 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.755415 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.755529 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.755640 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.755746 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.755859 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.755963 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.756059 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRemovesDuplicateGuardedBlocks (0.05s) === RUN TestRecoversFromBadNesting W0920 05:48:51.757491 6720 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0920 05:48:51.757798 6720 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0920 05:48:51.757847 6720 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0920 05:48:51.757898 6720 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0920 05:48:51.757935 6720 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0920 05:48:51.757984 6720 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0920 05:48:51.758034 6720 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0920 05:48:51.758104 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.758178 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.758223 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.758265 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.758311 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.758354 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.758396 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.758439 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.758484 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.758531 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.758575 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.758626 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.758679 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.758722 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.758764 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.758807 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.767370 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.767409 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.767485 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.767503 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.767573 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.767590 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.767675 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.767692 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.767760 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.767780 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.767842 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.767859 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.767959 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.767976 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.768046 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.768062 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.768124 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.768140 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.768197 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.768214 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.768280 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.768297 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.768360 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.768382 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.768471 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.768488 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.768546 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.768563 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.768621 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.768636 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.768698 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.768721 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.768781 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.768797 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.768853 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.768870 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.768935 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.768952 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.769030 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.769047 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.769105 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.769121 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.769182 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.769198 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.769259 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.769275 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.769342 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.769358 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.769437 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.769454 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.769519 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.769536 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.769605 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.769620 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.769685 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.769702 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.769776 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.769798 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.769867 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.769884 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.769966 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.769995 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.770101 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.770118 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.770175 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.770191 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.770248 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.770263 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.770329 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.770345 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.770460 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.770477 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.770543 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.770559 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.770707 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.771087 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.771163 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.771180 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.771251 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.771267 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.771325 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.771340 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.771401 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.771422 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.771504 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.771520 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.771588 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.771605 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.771671 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.771688 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.771748 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.771763 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.771820 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.771836 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.771917 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.771934 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.771992 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.772008 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.772063 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.772081 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.772148 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.772171 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.772250 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.772267 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.772325 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.772341 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.772398 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.772414 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.772510 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.772530 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.772597 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.772613 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.772675 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.772690 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.772751 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.772767 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.772834 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.772850 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.772919 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.772936 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.773005 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.773021 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.773076 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.773092 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.773153 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.773170 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.773239 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.773256 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.773322 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.773340 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.773404 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.773420 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.773493 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.773509 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.773581 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.773598 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.773656 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.773672 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.773736 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.773752 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.773822 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.773837 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.773907 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.773923 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.773978 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.773994 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.774050 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.774066 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.774134 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.774150 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.774217 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.774238 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.774310 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.774327 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.774382 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.774409 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.774504 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.774520 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.774580 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.774604 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.774664 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.774679 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.774734 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.774750 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.774825 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.774841 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.780907 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.780946 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.781017 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.781035 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.781110 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.781127 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.781188 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.781204 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.781273 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.781290 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.781350 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.781366 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.781434 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.781457 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.781522 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.781540 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.781602 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.781618 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.781689 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.781709 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.781769 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.781789 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.781849 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.781865 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.781927 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.781943 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.781997 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.782013 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.782074 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.782091 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0920 05:48:51.782153 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0920 05:48:51.782170 6720 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRecoversFromBadNesting (0.03s) PASS ok k8s.io/kops/protokube/pkg/gossip/dns/hosts 0.094s ? k8s.io/kops/protokube/pkg/gossip/dns/provider [no test files] ? k8s.io/kops/protokube/pkg/gossip/do [no test files] ? k8s.io/kops/protokube/pkg/gossip/hetzner [no test files] ? k8s.io/kops/protokube/pkg/gossip/memberlist [no test files] ? k8s.io/kops/protokube/pkg/gossip/mesh [no test files] ? k8s.io/kops/protokube/pkg/gossip/openstack [no test files] === RUN TestGetMetadataFromMetadataServiceReturnsNotFoundError --- PASS: TestGetMetadataFromMetadataServiceReturnsNotFoundError (0.00s) === RUN TestGetMetadataFromMetadataService --- PASS: TestGetMetadataFromMetadataService (0.00s) === RUN TestGetMetadataFromConfigDriveReturnsErrorWhenNoDeviceIsFound --- PASS: TestGetMetadataFromConfigDriveReturnsErrorWhenNoDeviceIsFound (0.00s) === RUN TestGetMetadataFromConfigDrive --- PASS: TestGetMetadataFromConfigDrive (0.00s) === RUN TestGetMetadataReturnsLastErrorWhenNoMetadataWasFound --- PASS: TestGetMetadataReturnsLastErrorWhenNoMetadataWasFound (0.00s) === RUN TestGetMetadataFromConfigDriveWhenItIsFirstInSearchOrder --- PASS: TestGetMetadataFromConfigDriveWhenItIsFirstInSearchOrder (0.00s) === RUN TestGetMetadataFromServiceEndpointWhenConfigDriveFails --- PASS: TestGetMetadataFromServiceEndpointWhenConfigDriveFails (0.00s) === RUN TestGetMetadataFromServiceEndpointWhenItIsFirstInSearchOrder --- PASS: TestGetMetadataFromServiceEndpointWhenItIsFirstInSearchOrder (0.00s) === RUN TestGetMetadataFromConfigDriveWhenServiceEndpointFails --- PASS: TestGetMetadataFromConfigDriveWhenServiceEndpointFails (0.00s) PASS ok k8s.io/kops/protokube/pkg/protokube 0.073s === RUN TestSerializeEmptyCluster --- PASS: TestSerializeEmptyCluster (0.03s) PASS ok k8s.io/kops/tests/codecs 0.122s ? k8s.io/kops/tests/fuzz [no test files] === RUN TestKopsUpgrades --- PASS: TestKopsUpgrades (0.01s) === RUN TestKubernetesUpgrades --- PASS: TestKubernetesUpgrades (0.00s) === RUN TestFindImage --- PASS: TestFindImage (0.00s) === RUN TestRecommendedKubernetesVersion --- PASS: TestRecommendedKubernetesVersion (0.00s) === RUN TestOrdering --- PASS: TestOrdering (0.00s) === RUN TestChannelsSelfConsistent --- PASS: TestChannelsSelfConsistent (0.02s) === RUN TestChannelImages === RUN TestChannelImages/stable-channel === RUN TestChannelImages/alpha-channel --- PASS: TestChannelImages (0.02s) --- PASS: TestChannelImages/stable-channel (0.00s) --- PASS: TestChannelImages/alpha-channel (0.02s) PASS ok k8s.io/kops/tests/integration/channel 0.120s ? k8s.io/kops/tests/integration/channel/simple [no test files] === RUN TestConversionMinimal === RUN TestConversionMinimal/legacy-v1alpha2-v1alpha2 === RUN TestConversionMinimal/v1alpha2-v1alpha3 === RUN TestConversionMinimal/v1alpha3-v1alpha2 --- PASS: TestConversionMinimal (0.08s) --- PASS: TestConversionMinimal/legacy-v1alpha2-v1alpha2 (0.03s) --- PASS: TestConversionMinimal/v1alpha2-v1alpha3 (0.02s) --- PASS: TestConversionMinimal/v1alpha3-v1alpha2 (0.03s) === RUN TestConversionAWS === RUN TestConversionAWS/v1alpha2-v1alpha3 === RUN TestConversionAWS/v1alpha3-v1alpha2 --- PASS: TestConversionAWS (0.02s) --- PASS: TestConversionAWS/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionAWS/v1alpha3-v1alpha2 (0.01s) === RUN TestConversionAzure === RUN TestConversionAzure/v1alpha2-v1alpha3 === RUN TestConversionAzure/v1alpha3-v1alpha2 --- PASS: TestConversionAzure (0.01s) --- PASS: TestConversionAzure/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionAzure/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionCanal === RUN TestConversionCanal/v1alpha2-v1alpha3 === RUN TestConversionCanal/v1alpha3-v1alpha2 --- PASS: TestConversionCanal (0.02s) --- PASS: TestConversionCanal/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionCanal/v1alpha3-v1alpha2 (0.02s) === RUN TestConversionCilium === RUN TestConversionCilium/v1alpha2-v1alpha3 === RUN TestConversionCilium/v1alpha3-v1alpha2 --- PASS: TestConversionCilium (0.04s) --- PASS: TestConversionCilium/v1alpha2-v1alpha3 (0.02s) --- PASS: TestConversionCilium/v1alpha3-v1alpha2 (0.01s) === RUN TestConversionDO === RUN TestConversionDO/v1alpha2-v1alpha3 === RUN TestConversionDO/v1alpha3-v1alpha2 --- PASS: TestConversionDO (0.02s) --- PASS: TestConversionDO/v1alpha2-v1alpha3 (0.01s) --- PASS: TestConversionDO/v1alpha3-v1alpha2 (0.01s) === RUN TestConversionGCE === RUN TestConversionGCE/v1alpha2-v1alpha3 === RUN TestConversionGCE/v1alpha3-v1alpha2 --- PASS: TestConversionGCE (0.03s) --- PASS: TestConversionGCE/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionGCE/v1alpha3-v1alpha2 (0.02s) === RUN TestConversionOpenstack === RUN TestConversionOpenstack/v1alpha2-v1alpha3 === RUN TestConversionOpenstack/v1alpha3-v1alpha2 --- PASS: TestConversionOpenstack (0.02s) --- PASS: TestConversionOpenstack/v1alpha2-v1alpha3 (0.01s) --- PASS: TestConversionOpenstack/v1alpha3-v1alpha2 (0.00s) PASS ok k8s.io/kops/tests/integration/conversion 0.316s ? k8s.io/kops/upup/models [no test files] === RUN Test_tryResourceAsString --- PASS: Test_tryResourceAsString (0.00s) === RUN Test_DryrunTarget_PrintReport --- PASS: Test_DryrunTarget_PrintReport (0.00s) === RUN TestWriteFile I0920 05:48:57.566897 7942 files.go:57] Writing file "/tmp/TestWriteFile907954061/001/SubDir/test1.tmp" I0920 05:48:57.567105 7942 files.go:113] Changing file mode for "/tmp/TestWriteFile907954061/001/SubDir/test1.tmp" to -rw-r--r-- --- PASS: TestWriteFile (0.00s) === RUN TestBigInt_Format --- PASS: TestBigInt_Format (0.00s) === RUN TestVFSCAStoreRoundTrip --- PASS: TestVFSCAStoreRoundTrip (0.02s) === RUN TestVFSCAStoreRoundTripWithVault vfs_castore_test.go:161: No vault dev token set --- SKIP: TestVFSCAStoreRoundTripWithVault (0.00s) === RUN TestAddItem === RUN TestAddItem/first_primary === RUN TestAddItem/first_secondary === RUN TestAddItem/firstBigSerial_primary === RUN TestAddItem/firstBigSerial_secondary === RUN TestAddItem/after_primary === RUN TestAddItem/after_secondary === RUN TestAddItem/bigSerialAfter_primary === RUN TestAddItem/bigSerialAfter_secondary === RUN TestAddItem/before_primary === RUN TestAddItem/before_secondary === RUN TestAddItem/first_certonly_primary === RUN TestAddItem/first_certonly_secondary === RUN TestAddItem/after_certonly_primary === RUN TestAddItem/after_certonly_secondary === RUN TestAddItem/before_certonly_primary === RUN TestAddItem/before_certonly_secondary --- PASS: TestAddItem (0.00s) --- PASS: TestAddItem/first_primary (0.00s) --- PASS: TestAddItem/first_secondary (0.00s) --- PASS: TestAddItem/firstBigSerial_primary (0.00s) --- PASS: TestAddItem/firstBigSerial_secondary (0.00s) --- PASS: TestAddItem/after_primary (0.00s) --- PASS: TestAddItem/after_secondary (0.00s) --- PASS: TestAddItem/bigSerialAfter_primary (0.00s) --- PASS: TestAddItem/bigSerialAfter_secondary (0.00s) --- PASS: TestAddItem/before_primary (0.00s) --- PASS: TestAddItem/before_secondary (0.00s) --- PASS: TestAddItem/first_certonly_primary (0.00s) --- PASS: TestAddItem/first_certonly_secondary (0.00s) --- PASS: TestAddItem/after_certonly_primary (0.00s) --- PASS: TestAddItem/after_certonly_secondary (0.00s) --- PASS: TestAddItem/before_certonly_primary (0.00s) --- PASS: TestAddItem/before_certonly_secondary (0.00s) PASS ok k8s.io/kops/upup/pkg/fi 0.095s === RUN TestBootstrapChannelBuilder_BuildTasks I0920 05:49:05.818359 9638 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:05.818606 9638 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:05.818657 9638 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:05.818701 9638 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:05.818748 9638 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:05.818795 9638 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:05.818813 9638 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:05.818840 9638 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:05.818872 9638 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:05.829681 9638 zones.go:93] ListHostedZonesPages { } I0920 05:49:05.908732 9638 zones.go:93] ListHostedZonesPages { } I0920 05:49:06.050141 9638 zones.go:93] ListHostedZonesPages { } I0920 05:49:06.122617 9638 zones.go:93] ListHostedZonesPages { } I0920 05:49:06.182644 9638 zones.go:93] ListHostedZonesPages { } I0920 05:49:06.251651 9638 zones.go:93] ListHostedZonesPages { } I0920 05:49:06.314996 9638 zones.go:93] ListHostedZonesPages { } I0920 05:49:06.385669 9638 zones.go:93] ListHostedZonesPages { } I0920 05:49:06.478302 9638 zones.go:93] ListHostedZonesPages { } I0920 05:49:06.855708 9638 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_BuildTasks (1.09s) === RUN TestBootstrapChannelBuilder_ServiceAccountIAM I0920 05:49:06.909025 9638 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:06.909188 9638 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:06.909252 9638 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:06.909292 9638 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:06.909326 9638 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:06.909364 9638 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:06.909390 9638 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:06.909414 9638 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:06.909433 9638 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:06.909458 9638 featureflag.go:163] Unknown FeatureFlag "+UseServiceAccountExternalPermissions" I0920 05:49:06.910707 9638 zones.go:93] ListHostedZonesPages { } I0920 05:49:06.965383 9638 featureflag.go:163] Unknown FeatureFlag "-UseServiceAccountExternalPermissions" --- PASS: TestBootstrapChannelBuilder_ServiceAccountIAM (0.06s) === RUN TestBootstrapChannelBuilder_AWSCloudController I0920 05:49:06.965612 9638 featureflag.go:160] FeatureFlag "ImageDigest"=false I0920 05:49:06.965777 9638 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:06.965827 9638 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0920 05:49:06.965859 9638 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0920 05:49:06.965898 9638 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0920 05:49:06.965924 9638 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0920 05:49:06.965942 9638 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0920 05:49:06.965963 9638 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0920 05:49:06.965979 9638 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0920 05:49:06.967141 9638 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_AWSCloudController (0.13s) === RUN TestContainerdVersionUrlHash === RUN TestContainerdVersionUrlHash/1.3.4-amd64 === RUN TestContainerdVersionUrlHash/1.3.4-arm64 === RUN TestContainerdVersionUrlHash/1.3.10-amd64 === RUN TestContainerdVersionUrlHash/1.3.10-arm64 === RUN TestContainerdVersionUrlHash/1.4.1-amd64 === RUN TestContainerdVersionUrlHash/1.4.1-arm64 === RUN TestContainerdVersionUrlHash/1.4.4-amd64 === RUN TestContainerdVersionUrlHash/1.4.4-arm64 --- PASS: TestContainerdVersionUrlHash (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-arm64 (0.00s) === RUN TestContainerdVersionUrl === RUN TestContainerdVersionUrl/1.4.1- === RUN TestContainerdVersionUrl/1.4.1-arm === RUN TestContainerdVersionUrl/-amd64 === RUN TestContainerdVersionUrl/-arm64 === RUN TestContainerdVersionUrl/1.1.1-amd64 === RUN TestContainerdVersionUrl/1.1.1-arm64 === RUN TestContainerdVersionUrl/1.3.5-amd64 === RUN TestContainerdVersionUrl/1.3.5-arm64 === RUN TestContainerdVersionUrl/1.3.4-amd64 === RUN TestContainerdVersionUrl/1.3.4-arm64 === RUN TestContainerdVersionUrl/1.4.1-amd64 === RUN TestContainerdVersionUrl/1.4.1-arm64 === RUN TestContainerdVersionUrl/1.4.3-amd64 === RUN TestContainerdVersionUrl/1.4.3-arm64 --- PASS: TestContainerdVersionUrl (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1- (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionUrl/-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionHash === RUN TestContainerdVersionHash/1.4.1- === RUN TestContainerdVersionHash/1.4.1-arm === RUN TestContainerdVersionHash/-amd64 === RUN TestContainerdVersionHash/-arm64 === RUN TestContainerdVersionHash/1.1.1-amd64 === RUN TestContainerdVersionHash/1.1.1-arm64 === RUN TestContainerdVersionHash/1.3.5-amd64 === RUN TestContainerdVersionHash/1.3.5-arm64 === RUN TestContainerdVersionHash/1.4.1-amd64 === RUN TestContainerdVersionHash/1.4.1-arm64 === RUN TestContainerdVersionHash/1.4.3-amd64 === RUN TestContainerdVersionHash/1.4.3-arm64 --- PASS: TestContainerdVersionHash (0.00s) --- PASS: TestContainerdVersionHash/1.4.1- (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionHash/-amd64 (0.00s) --- PASS: TestContainerdVersionHash/-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionsHashesAmd64 containerd_test.go:316: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesAmd64 (0.00s) === RUN TestContainerdVersionsHashesArm64 containerd_test.go:331: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesArm64 (0.00s) === RUN TestDeepValidate_OK --- PASS: TestDeepValidate_OK (0.00s) === RUN TestDeepValidate_NoNodeZones --- PASS: TestDeepValidate_NoNodeZones (0.00s) === RUN TestDeepValidate_NoMasterZones --- PASS: TestDeepValidate_NoMasterZones (0.00s) === RUN TestDeepValidate_BadZone deepvalidate_test.go:57: Zone validation not checked by DeepValidate --- SKIP: TestDeepValidate_BadZone (0.00s) === RUN TestDeepValidate_MixedRegion deepvalidate_test.go:69: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_MixedRegion (0.00s) === RUN TestDeepValidate_RegionAsZone deepvalidate_test.go:83: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_RegionAsZone (0.00s) === RUN TestDeepValidate_NotIncludedZone --- PASS: TestDeepValidate_NotIncludedZone (0.00s) === RUN TestDeepValidate_DuplicateZones --- PASS: TestDeepValidate_DuplicateZones (0.00s) === RUN TestDeepValidate_ExtraMasterZone --- PASS: TestDeepValidate_ExtraMasterZone (0.00s) === RUN TestDeepValidate_EvenEtcdClusterSize --- PASS: TestDeepValidate_EvenEtcdClusterSize (0.00s) === RUN TestDeepValidate_MissingEtcdMember --- PASS: TestDeepValidate_MissingEtcdMember (0.00s) === RUN TestPopulateClusterSpec_Proxy --- PASS: TestPopulateClusterSpec_Proxy (0.00s) === RUN TestPrecreateDNSNames --- PASS: TestPrecreateDNSNames (0.00s) === RUN TestDockerVersionUrlHash === RUN TestDockerVersionUrlHash/19.03.13-amd64 === RUN TestDockerVersionUrlHash/19.03.13-arm64 === RUN TestDockerVersionUrlHash/18.06.3-amd64 === RUN TestDockerVersionUrlHash/18.06.3-arm64 === RUN TestDockerVersionUrlHash/17.03.1-amd64 === RUN TestDockerVersionUrlHash/17.03.1-arm64 --- PASS: TestDockerVersionUrlHash (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-arm64 (0.00s) === RUN TestDockerVersionUrl === RUN TestDockerVersionUrl/19.03.13- === RUN TestDockerVersionUrl/19.03.13-arm === RUN TestDockerVersionUrl/-amd64 === RUN TestDockerVersionUrl/-arm64 === RUN TestDockerVersionUrl/18.06.3-amd64 === RUN TestDockerVersionUrl/18.06.3-arm64 === RUN TestDockerVersionUrl/19.03.13-amd64 === RUN TestDockerVersionUrl/19.03.13-arm64 --- PASS: TestDockerVersionUrl (0.00s) --- PASS: TestDockerVersionUrl/19.03.13- (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm (0.00s) --- PASS: TestDockerVersionUrl/-amd64 (0.00s) --- PASS: TestDockerVersionUrl/-arm64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm64 (0.00s) === RUN TestDockerVersionHash === RUN TestDockerVersionHash/19.03.13- === RUN TestDockerVersionHash/19.03.13-arm === RUN TestDockerVersionHash/-amd64 === RUN TestDockerVersionHash/-arm64 === RUN TestDockerVersionHash/1.1.1-amd64 === RUN TestDockerVersionHash/1.1.1-arm64 === RUN TestDockerVersionHash/19.03.13-amd64 === RUN TestDockerVersionHash/19.03.13-arm64 --- PASS: TestDockerVersionHash (0.00s) --- PASS: TestDockerVersionHash/19.03.13- (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm (0.00s) --- PASS: TestDockerVersionHash/-amd64 (0.00s) --- PASS: TestDockerVersionHash/-arm64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm64 (0.00s) === RUN TestDockerVersionsHashesAmd64 docker_test.go:245: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesAmd64 (0.00s) === RUN TestDockerVersionsHashesArm64 docker_test.go:260: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesArm64 (0.00s) === RUN Test_FindCNIAssetFromEnvironmentVariable --- PASS: Test_FindCNIAssetFromEnvironmentVariable (0.00s) === RUN Test_FindCNIAssetFromDefaults118 --- PASS: Test_FindCNIAssetFromDefaults118 (0.07s) === RUN Test_FindCNIAssetFromDefaults122 --- PASS: Test_FindCNIAssetFromDefaults122 (0.01s) === RUN TestRemoveSharedPrefix --- PASS: TestRemoveSharedPrefix (0.00s) === RUN TestCreateEtcdCluster --- PASS: TestCreateEtcdCluster (0.00s) === RUN TestSetupNetworking W0920 05:49:07.203409 9638 new_cluster.go:1087] flannel UDP mode is not recommended; consider flannel-vxlan instead --- PASS: TestSetupNetworking (0.01s) === RUN TestPopulateCluster_Default_NoError --- PASS: TestPopulateCluster_Default_NoError (0.00s) === RUN TestPopulateCluster_Subnets === RUN TestPopulateCluster_Subnets/100.64.0.0/10 === RUN TestPopulateCluster_Subnets/10.0.0.0/9 === RUN TestPopulateCluster_Subnets/10.0.0.0/8 === RUN TestPopulateCluster_Subnets/::/0 --- PASS: TestPopulateCluster_Subnets (0.01s) --- PASS: TestPopulateCluster_Subnets/100.64.0.0/10 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/9 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/8 (0.00s) --- PASS: TestPopulateCluster_Subnets/::/0 (0.00s) === RUN TestPopulateCluster_Docker_Spec --- PASS: TestPopulateCluster_Docker_Spec (0.00s) === RUN TestPopulateCluster_StorageDefault --- PASS: TestPopulateCluster_StorageDefault (0.00s) === RUN TestPopulateCluster_Custom_CIDR --- PASS: TestPopulateCluster_Custom_CIDR (0.00s) === RUN TestPopulateCluster_IsolateMasters --- PASS: TestPopulateCluster_IsolateMasters (0.00s) === RUN TestPopulateCluster_IsolateMastersFalse --- PASS: TestPopulateCluster_IsolateMastersFalse (0.00s) === RUN TestPopulateCluster_Name_Required --- PASS: TestPopulateCluster_Name_Required (0.00s) === RUN TestPopulateCluster_Zone_Required --- PASS: TestPopulateCluster_Zone_Required (0.00s) === RUN TestPopulateCluster_NetworkCIDR_Required --- PASS: TestPopulateCluster_NetworkCIDR_Required (0.00s) === RUN TestPopulateCluster_NonMasqueradeCIDR_Required --- PASS: TestPopulateCluster_NonMasqueradeCIDR_Required (0.00s) === RUN TestPopulateCluster_CloudProvider_Required --- PASS: TestPopulateCluster_CloudProvider_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidNil_Required --- PASS: TestPopulateCluster_TopologyInvalidNil_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidValue_Required --- PASS: TestPopulateCluster_TopologyInvalidValue_Required (0.00s) === RUN TestPopulateCluster_BastionInvalidMatchingValues_Required --- PASS: TestPopulateCluster_BastionInvalidMatchingValues_Required (0.00s) === RUN TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required --- PASS: TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required (0.00s) === RUN TestPopulateCluster_APIServerCount --- PASS: TestPopulateCluster_APIServerCount (0.00s) === RUN TestPopulateCluster_AnonymousAuth --- PASS: TestPopulateCluster_AnonymousAuth (0.00s) === RUN TestPopulateCluster_DockerVersion --- PASS: TestPopulateCluster_DockerVersion (0.01s) === RUN TestPopulateCluster_KubeController_High_Enough_Version --- PASS: TestPopulateCluster_KubeController_High_Enough_Version (0.00s) === RUN TestPopulateInstanceGroup_Name_Required --- PASS: TestPopulateInstanceGroup_Name_Required (0.00s) === RUN TestPopulateInstanceGroup_Role_Required --- PASS: TestPopulateInstanceGroup_Role_Required (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision --- PASS: TestPopulateInstanceGroup_AddTaintsCollision (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision3 --- PASS: TestPopulateInstanceGroup_AddTaintsCollision3 (0.00s) === RUN TestPopulateInstanceGroup_AddTaints --- PASS: TestPopulateInstanceGroup_AddTaints (0.00s) === RUN TestMachineArchitecture === RUN TestMachineArchitecture/t2.micro-amd64 === RUN TestMachineArchitecture/t3.micro-amd64 === RUN TestMachineArchitecture/a1.large-arm64 --- PASS: TestMachineArchitecture (0.00s) --- PASS: TestMachineArchitecture/t2.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/t3.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/a1.large-arm64 (0.00s) === RUN TestRuncVersionUrlHash === RUN TestRuncVersionUrlHash/1.100.0-amd64 === RUN TestRuncVersionUrlHash/1.100.0-arm64 === RUN TestRuncVersionUrlHash/1.1.0-amd64 === RUN TestRuncVersionUrlHash/1.1.0-arm64 --- PASS: TestRuncVersionUrlHash (0.00s) --- PASS: TestRuncVersionUrlHash/1.100.0-amd64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.100.0-arm64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.1.0-arm64 (0.00s) === RUN TestRuncVersionUrl === RUN TestRuncVersionUrl/1.1.0- === RUN TestRuncVersionUrl/1.1.0-arm === RUN TestRuncVersionUrl/-amd64 === RUN TestRuncVersionUrl/-arm64 === RUN TestRuncVersionUrl/1.0.0-amd64 === RUN TestRuncVersionUrl/1.0.0-arm64 === RUN TestRuncVersionUrl/1.1.0-amd64 === RUN TestRuncVersionUrl/1.1.0-arm64 --- PASS: TestRuncVersionUrl (0.00s) --- PASS: TestRuncVersionUrl/1.1.0- (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-arm (0.00s) --- PASS: TestRuncVersionUrl/-amd64 (0.00s) --- PASS: TestRuncVersionUrl/-arm64 (0.00s) --- PASS: TestRuncVersionUrl/1.0.0-amd64 (0.00s) --- PASS: TestRuncVersionUrl/1.0.0-arm64 (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-arm64 (0.00s) === RUN TestRuncVersionHash === RUN TestRuncVersionHash/1.1.0- === RUN TestRuncVersionHash/1.1.0-arm === RUN TestRuncVersionHash/-amd64 === RUN TestRuncVersionHash/-arm64 === RUN TestRuncVersionHash/1.0.0-amd64 === RUN TestRuncVersionHash/1.0.0-arm64 === RUN TestRuncVersionHash/1.100.0-amd64 === RUN TestRuncVersionHash/1.100.0-arm64 === RUN TestRuncVersionHash/1.1.0-amd64 === RUN TestRuncVersionHash/1.1.0-arm64 --- PASS: TestRuncVersionHash (0.00s) --- PASS: TestRuncVersionHash/1.1.0- (0.00s) --- PASS: TestRuncVersionHash/1.1.0-arm (0.00s) --- PASS: TestRuncVersionHash/-amd64 (0.00s) --- PASS: TestRuncVersionHash/-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.0.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.0.0-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.100.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.100.0-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.1.0-arm64 (0.00s) === RUN TestRuncVersionsHashesAmd64 runc_test.go:240: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesAmd64 (0.00s) === RUN TestRuncVersionsHashesArm64 runc_test.go:255: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesArm64 (0.00s) === RUN Test_AssignSubnets I0920 05:49:07.259127 9638 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0920 05:49:07.259173 9638 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0920 05:49:07.259193 9638 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I0920 05:49:07.259229 9638 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I0920 05:49:07.259251 9638 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet a I0920 05:49:07.259289 9638 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I0920 05:49:07.259324 9638 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet b I0920 05:49:07.259357 9638 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0920 05:49:07.259379 9638 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0920 05:49:07.259411 9638 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0920 05:49:07.259432 9638 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0920 05:49:07.259463 9638 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0920 05:49:07.259506 9638 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0920 05:49:07.259540 9638 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0920 05:49:07.259560 9638 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I0920 05:49:07.259576 9638 subnets.go:185] Assigned CIDR 10.96.0.0/11 to subnet b I0920 05:49:07.259594 9638 subnets.go:185] Assigned CIDR 10.128.0.0/11 to subnet c I0920 05:49:07.259610 9638 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet c I0920 05:49:07.259628 9638 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0920 05:49:07.259647 9638 subnets.go:199] Assigned CIDR 10.4.0.0/14 to subnet b I0920 05:49:07.259665 9638 subnets.go:199] Assigned CIDR 10.8.0.0/14 to subnet c --- PASS: Test_AssignSubnets (0.00s) === RUN Test_TemplateFunctions_CloudControllerConfigArgv === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller (0.00s) === RUN Test_KarpenterInstanceTypes I0920 05:49:07.261425 9638 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: Test_KarpenterInstanceTypes (0.00s) === RUN Test_BuildMirroredAsset === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup --- PASS: Test_BuildMirroredAsset (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup (0.00s) === RUN TestValidateFull_Default_Validates --- PASS: TestValidateFull_Default_Validates (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_NoDot --- PASS: TestValidateFull_ClusterName_InvalidDNS_NoDot (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_Invalid --- PASS: TestValidateFull_ClusterName_InvalidDNS_Invalid (0.00s) === RUN TestValidateFull_ClusterName_Required --- PASS: TestValidateFull_ClusterName_Required (0.00s) === RUN TestValidateFull_UpdatePolicy_Valid === RUN TestValidateFull_UpdatePolicy_Valid/missing === RUN TestValidateFull_UpdatePolicy_Valid/automatic === RUN TestValidateFull_UpdatePolicy_Valid/external --- PASS: TestValidateFull_UpdatePolicy_Valid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/missing (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/automatic (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/external (0.00s) === RUN TestValidateFull_UpdatePolicy_Invalid === RUN TestValidateFull_UpdatePolicy_Invalid/empty === RUN TestValidateFull_UpdatePolicy_Invalid/populated --- PASS: TestValidateFull_UpdatePolicy_Invalid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/empty (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/populated (0.00s) === RUN Test_Validate_Kubenet_With_14 --- PASS: Test_Validate_Kubenet_With_14 (0.00s) === RUN TestValidate_ClusterName_Import --- PASS: TestValidate_ClusterName_Import (0.00s) === RUN TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity --- PASS: TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup 1.542s === RUN TestGetASGTagsToDelete --- PASS: TestGetASGTagsToDelete (0.00s) === RUN TestProcessCompare --- PASS: TestProcessCompare (0.00s) === RUN TestAutoscalingGroupTerraformRender I0920 05:49:04.280983 9563 target.go:112] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender2444810083/001 I0920 05:49:04.283525 9563 target.go:112] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender2444810083/001 --- PASS: TestAutoscalingGroupTerraformRender (0.00s) === RUN TestAutoscalingGroupCloudformationRender I0920 05:49:04.284618 9563 target.go:168] Cloudformation output is in /tmp/TestAutoscalingGroupCloudformationRender1433446487/001 --- PASS: TestAutoscalingGroupCloudformationRender (0.00s) === RUN TestGetEBSVolumeTagsToDelete --- PASS: TestGetEBSVolumeTagsToDelete (0.00s) === RUN TestSharedEgressOnlyInternetGatewayDoesNotRename I0920 05:49:04.285275 9563 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0920 05:49:04.285351 9563 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0920 05:49:04.285427 9563 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { TagSpecifications: [{ ResourceType: "egress-only-internet-gateway", Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] }], VpcId: "vpc-1" } I0920 05:49:04.285614 9563 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0920 05:49:04.285706 9563 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:04.285770 9563 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0920 05:49:04.285789 9563 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0920 05:49:04.285929 9563 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0920 05:49:04.285981 9563 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0920 05:49:04.286027 9563 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0920 05:49:04.286107 9563 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0920 05:49:04.286198 9563 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0920 05:49:04.286320 9563 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0920 05:49:04.286375 9563 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:04.286420 9563 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0920 05:49:04.286435 9563 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0920 05:49:04.286472 9563 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0920 05:49:04.286519 9563 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0920 05:49:04.286575 9563 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedEgressOnlyInternetGatewayDoesNotRename (0.00s) === RUN TestElasticIPCreate I0920 05:49:04.286876 9563 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0920 05:49:04.286923 9563 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0920 05:49:04.287024 9563 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0920 05:49:04.287063 9563 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0920 05:49:04.287089 9563 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:04.287115 9563 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0920 05:49:04.287235 9563 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0920 05:49:04.287414 9563 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0920 05:49:04.287468 9563 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:04.287492 9563 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:04.287517 9563 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0920 05:49:04.287567 9563 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:04.287701 9563 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "eip1" }] }] } I0920 05:49:04.287764 9563 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:04.287788 9563 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedElasticIp", Value: "192.0.2.1" },{ Key: "AssociatedElasticIpAllocationId", Value: "eipalloc-1" }] } I0920 05:49:04.287824 9563 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0920 05:49:04.287968 9563 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0920 05:49:04.288013 9563 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0920 05:49:04.288056 9563 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0920 05:49:04.288071 9563 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0920 05:49:04.288107 9563 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:04.288134 9563 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0920 05:49:04.288170 9563 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0920 05:49:04.288223 9563 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:04.288248 9563 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0920 05:49:04.288284 9563 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:04.288328 9563 address.go:120] DescribeAddresses: { Filters: [{ Name: "public-ip", Values: ["192.0.2.1"] }] } I0920 05:49:04.288363 9563 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0920 05:49:04.288403 9563 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestElasticIPCreate (0.00s) === RUN TestSharedInternetGatewayDoesNotRename I0920 05:49:04.288550 9563 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0920 05:49:04.288592 9563 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0920 05:49:04.288615 9563 internetgateways.go:66] CreateInternetGateway: { } I0920 05:49:04.288632 9563 tags.go:42] CreateTags { Resources: ["igw-1"], Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] } I0920 05:49:04.288744 9563 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0920 05:49:04.288786 9563 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:04.288822 9563 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0920 05:49:04.288838 9563 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0920 05:49:04.288903 9563 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0920 05:49:04.288929 9563 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0920 05:49:04.288961 9563 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0920 05:49:04.289014 9563 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0920 05:49:04.289085 9563 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0920 05:49:04.289200 9563 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0920 05:49:04.289249 9563 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:04.289289 9563 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0920 05:49:04.289305 9563 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0920 05:49:04.289336 9563 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0920 05:49:04.289371 9563 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0920 05:49:04.289424 9563 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedInternetGatewayDoesNotRename (0.00s) === RUN TestLaunchTemplateCloudformationRender I0920 05:49:04.290291 9563 target.go:168] Cloudformation output is in /tmp/TestLaunchTemplateCloudformationRender254880879/001 I0920 05:49:04.290540 9563 target.go:168] Cloudformation output is in /tmp/TestLaunchTemplateCloudformationRender254880879/001 --- PASS: TestLaunchTemplateCloudformationRender (0.00s) === RUN TestLaunchTemplateTerraformRender I0920 05:49:04.292417 9563 target.go:112] Terraform output is in /tmp/TestLaunchTemplateTerraformRender48631472/001 I0920 05:49:04.294207 9563 target.go:112] Terraform output is in /tmp/TestLaunchTemplateTerraformRender48631472/001 --- PASS: TestLaunchTemplateTerraformRender (0.00s) === RUN TestParseRemovalRule --- PASS: TestParseRemovalRule (0.00s) === RUN TestPortRemovalRule --- PASS: TestPortRemovalRule (0.00s) === RUN TestPortRemovalRule_Zero --- PASS: TestPortRemovalRule_Zero (0.00s) === RUN TestSecurityGroupCreate I0920 05:49:04.294784 9563 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0920 05:49:04.294843 9563 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0920 05:49:04.294953 9563 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0920 05:49:04.294999 9563 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0920 05:49:04.295025 9563 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:04.295050 9563 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0920 05:49:04.295117 9563 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0920 05:49:04.295320 9563 securitygroups.go:40] CreateSecurityGroup: { Description: "Description", GroupName: "sg1", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "sg1" }] }], VpcId: "vpc-1" } I0920 05:49:04.295372 9563 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0920 05:49:04.295405 9563 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0920 05:49:04.295526 9563 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0920 05:49:04.295572 9563 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0920 05:49:04.295616 9563 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0920 05:49:04.295633 9563 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0920 05:49:04.295658 9563 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:04.295682 9563 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0920 05:49:04.295724 9563 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0920 05:49:04.295794 9563 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSecurityGroupCreate (0.00s) === RUN Test_Subnet_ValidateRequired --- PASS: Test_Subnet_ValidateRequired (0.00s) === RUN Test_Subnet_CannotChangeSubnet --- PASS: Test_Subnet_CannotChangeSubnet (0.00s) === RUN TestSubnetCreate I0920 05:49:04.296156 9563 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0920 05:49:04.296200 9563 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0920 05:49:04.296295 9563 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0920 05:49:04.296332 9563 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0920 05:49:04.296353 9563 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:04.296380 9563 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0920 05:49:04.296420 9563 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0920 05:49:04.296521 9563 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0920 05:49:04.296561 9563 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:04.296583 9563 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:04.296606 9563 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0920 05:49:04.296768 9563 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0920 05:49:04.296812 9563 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0920 05:49:04.296854 9563 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0920 05:49:04.296869 9563 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0920 05:49:04.296893 9563 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:04.296916 9563 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0920 05:49:04.296951 9563 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0920 05:49:04.297008 9563 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:04.297034 9563 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSubnetCreate (0.00s) === RUN TestSubnetCreateIPv6 I0920 05:49:04.297301 9563 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0920 05:49:04.297344 9563 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0920 05:49:04.297426 9563 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0920 05:49:04.297464 9563 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0920 05:49:04.297487 9563 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:04.297510 9563 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0920 05:49:04.297551 9563 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0920 05:49:04.297743 9563 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0920 05:49:04.297705 9563 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:04.297904 9563 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0920 05:49:04.297961 9563 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:04.297993 9563 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:04.298020 9563 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0920 05:49:04.298160 9563 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0920 05:49:04.298204 9563 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0920 05:49:04.298250 9563 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0920 05:49:04.298266 9563 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0920 05:49:04.298290 9563 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:04.298314 9563 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0920 05:49:04.298351 9563 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0920 05:49:04.298408 9563 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:04.298429 9563 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:04.298475 9563 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6 (0.00s) === RUN TestSubnetCreateIPv6NetNum I0920 05:49:04.298723 9563 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0920 05:49:04.298766 9563 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0920 05:49:04.298873 9563 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0920 05:49:04.298913 9563 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0920 05:49:04.298935 9563 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:04.298958 9563 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0920 05:49:04.298994 9563 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:04.299066 9563 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0920 05:49:04.299092 9563 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0920 05:49:04.299260 9563 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0920 05:49:04.299307 9563 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:04.299331 9563 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:04.299354 9563 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0920 05:49:04.299510 9563 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0920 05:49:04.299554 9563 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0920 05:49:04.299596 9563 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0920 05:49:04.299611 9563 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0920 05:49:04.299636 9563 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:04.299658 9563 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0920 05:49:04.299694 9563 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0920 05:49:04.299785 9563 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:04.299814 9563 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:04.299859 9563 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6NetNum (0.00s) === RUN TestSharedSubnetCreateDoesNotCreateNew I0920 05:49:04.300030 9563 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "ExistingVPC" }] }] } I0920 05:49:04.300099 9563 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "ExistingSubnet" }] }], VpcId: "vpc-1" } I0920 05:49:04.300223 9563 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0920 05:49:04.300266 9563 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:04.300303 9563 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0920 05:49:04.300318 9563 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0920 05:49:04.300382 9563 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0920 05:49:04.300408 9563 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0920 05:49:04.300440 9563 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0920 05:49:04.300473 9563 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:04.300566 9563 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0920 05:49:04.300593 9563 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0920 05:49:04.300623 9563 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0920 05:49:04.300747 9563 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0920 05:49:04.300788 9563 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:04.300824 9563 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0920 05:49:04.300839 9563 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0920 05:49:04.300869 9563 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0920 05:49:04.300909 9563 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0920 05:49:04.300969 9563 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedSubnetCreateDoesNotCreateNew (0.00s) === RUN TestVPCCreate I0920 05:49:04.301185 9563 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0920 05:49:04.301235 9563 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0920 05:49:04.301326 9563 vpcs.go:89] CreateVpc: { CidrBlock: "172.21.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0920 05:49:04.301362 9563 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0920 05:49:04.301384 9563 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:04.301406 9563 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0920 05:49:04.301504 9563 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0920 05:49:04.301550 9563 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0920 05:49:04.301592 9563 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0920 05:49:04.301609 9563 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0920 05:49:04.301631 9563 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0920 05:49:04.301652 9563 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestVPCCreate (0.00s) === RUN Test4758 --- PASS: Test4758 (0.00s) === RUN TestSharedVPCAdditionalCIDR I0920 05:49:04.301822 9563 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "172.22.0.0/16", VpcId: "vpc-1" } I0920 05:49:04.301944 9563 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0920 05:49:04.301986 9563 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc-1"] }] } I0920 05:49:04.302026 9563 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0920 05:49:04.302041 9563 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0920 05:49:04.302083 9563 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestSharedVPCAdditionalCIDR (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awstasks 0.063s === RUN TestValidateRegion --- PASS: TestValidateRegion (0.00s) === RUN TestFindRegion --- PASS: TestFindRegion (0.00s) === RUN TestEC2TagSpecification === RUN TestEC2TagSpecification/No_tags === RUN TestEC2TagSpecification/simple_tag --- PASS: TestEC2TagSpecification (0.00s) --- PASS: TestEC2TagSpecification/No_tags (0.00s) --- PASS: TestEC2TagSpecification/simple_tag (0.00s) === RUN Test_GetResourceName32 --- PASS: Test_GetResourceName32 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awsup 0.044s === RUN TestZoneToLocation === RUN TestZoneToLocation/test_case_0 === RUN TestZoneToLocation/test_case_1 === RUN TestZoneToLocation/test_case_2 --- PASS: TestZoneToLocation (0.00s) --- PASS: TestZoneToLocation/test_case_0 (0.00s) --- PASS: TestZoneToLocation/test_case_1 (0.00s) --- PASS: TestZoneToLocation/test_case_2 (0.00s) === RUN TestZoneToAvailabilityZoneNumber === RUN TestZoneToAvailabilityZoneNumber/test_case_0 === RUN TestZoneToAvailabilityZoneNumber/test_case_1 === RUN TestZoneToAvailabilityZoneNumber/test_case_2 --- PASS: TestZoneToAvailabilityZoneNumber (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_0 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_1 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_2 (0.00s) === RUN TestFindEtcdStatus --- PASS: TestFindEtcdStatus (0.00s) === RUN TestGetCloudGroups --- PASS: TestGetCloudGroups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azure 0.107s === RUN TestDiskRenderAzure I0920 05:49:00.717376 8945 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRenderAzure (0.00s) === RUN TestDiskFind --- PASS: TestDiskFind (0.00s) === RUN TestDiskRun I0920 05:49:00.717818 8945 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRun (0.00s) === RUN TestDiskCheckChanges === RUN TestDiskCheckChanges/test_case_0 === RUN TestDiskCheckChanges/test_case_1 === RUN TestDiskCheckChanges/test_case_2 === RUN TestDiskCheckChanges/test_case_3 --- PASS: TestDiskCheckChanges (0.00s) --- PASS: TestDiskCheckChanges/test_case_0 (0.00s) --- PASS: TestDiskCheckChanges/test_case_1 (0.00s) --- PASS: TestDiskCheckChanges/test_case_2 (0.00s) --- PASS: TestDiskCheckChanges/test_case_3 (0.00s) === RUN TestLoadBalancerRenderAzure I0920 05:49:00.718107 8945 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRenderAzure (0.00s) === RUN TestLoadBalancerFind loadbalancer_test.go:122: *azuretasks.LoadBalancer {"Name":"loadbalancer","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Subnet":{"Name":"subnet","Lifecycle":"","ResourceGroup":null,"VirtualNetwork":null,"CIDR":null,"Shared":null},"External":true,"Tags":null,"ForAPIServer":false} --- PASS: TestLoadBalancerFind (0.00s) === RUN TestLoadBalancerRun I0920 05:49:00.718778 8945 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRun (0.00s) === RUN TestLoadBalancerCheckChanges === RUN TestLoadBalancerCheckChanges/test_case_0 === RUN TestLoadBalancerCheckChanges/test_case_1 === RUN TestLoadBalancerCheckChanges/test_case_2 === RUN TestLoadBalancerCheckChanges/test_case_3 --- PASS: TestLoadBalancerCheckChanges (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_0 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_1 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_2 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_3 (0.00s) === RUN TestPublicIPAddressRenderAzure I0920 05:49:00.719092 8945 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRenderAzure (0.00s) === RUN TestPublicIPAddressFind publicipaddress_test.go:100: *azuretasks.PublicIPAddress {"Name":"publicIPAddress","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Tags":null} --- PASS: TestPublicIPAddressFind (0.00s) === RUN TestPublicIPAddressRun I0920 05:49:00.719394 8945 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRun (0.00s) === RUN TestPublicIPAddressCheckChanges === RUN TestPublicIPAddressCheckChanges/test_case_0 === RUN TestPublicIPAddressCheckChanges/test_case_1 === RUN TestPublicIPAddressCheckChanges/test_case_2 === RUN TestPublicIPAddressCheckChanges/test_case_3 --- PASS: TestPublicIPAddressCheckChanges (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_0 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_1 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_2 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_3 (0.00s) === RUN TestResourceGroupRenderAzure I0920 05:49:00.719645 8945 resourcegroup.go:103] Creating a new Resource Group with name: rg I0920 05:49:00.719661 8945 resourcegroup.go:105] Updating a Resource Group with name: rg --- PASS: TestResourceGroupRenderAzure (0.00s) === RUN TestResourceGroupFind --- PASS: TestResourceGroupFind (0.00s) === RUN TestResourceGroupRun I0920 05:49:00.719906 8945 resourcegroup.go:103] Creating a new Resource Group with name: rg --- PASS: TestResourceGroupRun (0.00s) === RUN TestResourceGroupCheckChanges === RUN TestResourceGroupCheckChanges/test_case_0 === RUN TestResourceGroupCheckChanges/test_case_1 === RUN TestResourceGroupCheckChanges/test_case_2 === RUN TestResourceGroupCheckChanges/test_case_3 --- PASS: TestResourceGroupCheckChanges (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_0 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_1 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_2 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_3 (0.00s) === RUN TestRoleAssignmentRenderAzure --- PASS: TestRoleAssignmentRenderAzure (0.00s) === RUN TestRoleAssignmentFind --- PASS: TestRoleAssignmentFind (0.00s) === RUN TestRoleAssignmentFind_NoPrincipalID --- PASS: TestRoleAssignmentFind_NoPrincipalID (0.00s) === RUN TestRoleAssignmentCheckChanges === RUN TestRoleAssignmentCheckChanges/test_case_0 === RUN TestRoleAssignmentCheckChanges/test_case_1 === RUN TestRoleAssignmentCheckChanges/test_case_2 === RUN TestRoleAssignmentCheckChanges/test_case_3 --- PASS: TestRoleAssignmentCheckChanges (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_0 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_1 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_2 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_3 (0.00s) === RUN TestSubnetRenderAzure I0920 05:49:00.720396 8945 subnet.go:106] Creating a new Subnet with name: vnet --- PASS: TestSubnetRenderAzure (0.00s) === RUN TestSubnetFind --- PASS: TestSubnetFind (0.00s) === RUN TestSubnetCheckChanges === RUN TestSubnetCheckChanges/test_case_0 === RUN TestSubnetCheckChanges/test_case_1 === RUN TestSubnetCheckChanges/test_case_2 === RUN TestSubnetCheckChanges/test_case_3 --- PASS: TestSubnetCheckChanges (0.00s) --- PASS: TestSubnetCheckChanges/test_case_0 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_1 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_2 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_3 (0.00s) === RUN TestVirtualNetworkRenderAzure I0920 05:49:00.720607 8945 virtualnetwork.go:116] Creating a new Virtual Network with name: vnet --- PASS: TestVirtualNetworkRenderAzure (0.00s) === RUN TestVirtualNetworkFind --- PASS: TestVirtualNetworkFind (0.00s) === RUN TestVirtualNetworkRun I0920 05:49:00.720889 8945 virtualnetwork.go:116] Creating a new Virtual Network with name: rg --- PASS: TestVirtualNetworkRun (0.00s) === RUN TestVirtualNetworkCheckChanges === RUN TestVirtualNetworkCheckChanges/test_case_0 === RUN TestVirtualNetworkCheckChanges/test_case_1 === RUN TestVirtualNetworkCheckChanges/test_case_2 === RUN TestVirtualNetworkCheckChanges/test_case_3 --- PASS: TestVirtualNetworkCheckChanges (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_0 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_1 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_2 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_3 (0.00s) === RUN TestSubnetIDParse --- PASS: TestSubnetIDParse (0.00s) === RUN TestLoadBalancerIDParse --- PASS: TestLoadBalancerIDParse (0.00s) === RUN TestVMScaleSetRenderAzure I0920 05:49:00.721164 8945 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRenderAzure (0.00s) === RUN TestVMScaleSetFind --- PASS: TestVMScaleSetFind (0.00s) === RUN TestVMScaleSetRun I0920 05:49:00.721454 8945 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRun (0.00s) === RUN TestVMScaleSetCheckChanges === RUN TestVMScaleSetCheckChanges/test_case_0 === RUN TestVMScaleSetCheckChanges/test_case_1 === RUN TestVMScaleSetCheckChanges/test_case_2 === RUN TestVMScaleSetCheckChanges/test_case_3 --- PASS: TestVMScaleSetCheckChanges (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_0 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_1 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_2 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_3 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azuretasks 0.048s ? k8s.io/kops/upup/pkg/fi/cloudup/bootstrapchannelbuilder [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/cloudformation [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/do [no test files] === RUN Test_Find === RUN Test_Find/successfully_found_volume === RUN Test_Find/no_volume_found === RUN Test_Find/error_from_server --- PASS: Test_Find (0.00s) --- PASS: Test_Find/successfully_found_volume (0.00s) --- PASS: Test_Find/no_volume_found (0.00s) --- PASS: Test_Find/error_from_server (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/dotasks 0.085s ? k8s.io/kops/upup/pkg/fi/cloudup/gce [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcediscovery [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcemetadata [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmsigner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmverifier [no test files] === RUN TestProjectIAMBinding I0920 05:49:03.567411 9518 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0920 05:49:03.568054 9518 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0920 05:49:03.568212 9518 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0920 05:49:03.568736 9518 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0920 05:49:03.568893 9518 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0920 05:49:03.569057 9518 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestProjectIAMBinding (0.00s) === RUN TestServiceAccount I0920 05:49:03.569294 9518 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0920 05:49:03.569426 9518 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0920 05:49:03.569547 9518 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0920 05:49:03.569693 9518 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0920 05:49:03.570204 9518 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0920 05:49:03.570322 9518 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestServiceAccount (0.00s) === RUN TestStorageBucketIAM I0920 05:49:03.570601 9518 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0920 05:49:03.570898 9518 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0920 05:49:03.571037 9518 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0920 05:49:03.571420 9518 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0920 05:49:03.571553 9518 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0920 05:49:03.571723 9518 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestStorageBucketIAM (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/gcetasks 0.057s ? k8s.io/kops/upup/pkg/fi/cloudup/hetzner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/hetznertasks [no test files] === RUN Test_OpenstackCloud_GetApiIngressStatus === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled --- PASS: Test_OpenstackCloud_GetApiIngressStatus (0.01s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set (0.01s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstack 0.039s === RUN TestFilterPortsReturnsAllPorts --- PASS: TestFilterPortsReturnsAllPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPort --- PASS: TestFilterPortsReturnsOnlyTaggedPort (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPorts --- PASS: TestFilterPortsReturnsOnlyTaggedPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPortsWithLongClustername --- PASS: TestFilterPortsReturnsOnlyTaggedPortsWithLongClustername (0.00s) === RUN Test_Port_GetDependencies --- PASS: Test_Port_GetDependencies (0.00s) === RUN Test_NewPortTaskFromCloud === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found === RUN Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups --- PASS: Test_NewPortTaskFromCloud (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found (0.00s) --- PASS: Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups (0.00s) === RUN Test_Port_Find === RUN Test_Port_Find/nothing_found === RUN Test_Port_Find/port_found_no_tags === RUN Test_Port_Find/port_found_with_tags === RUN Test_Port_Find/multiple_ports_found === RUN Test_Port_Find/error_listing_ports --- PASS: Test_Port_Find (0.00s) --- PASS: Test_Port_Find/nothing_found (0.00s) --- PASS: Test_Port_Find/port_found_no_tags (0.00s) --- PASS: Test_Port_Find/port_found_with_tags (0.00s) --- PASS: Test_Port_Find/multiple_ports_found (0.00s) --- PASS: Test_Port_Find/error_listing_ports (0.00s) === RUN Test_Port_CheckChanges === RUN Test_Port_CheckChanges/actual_nil_all_required_fields_set === RUN Test_Port_CheckChanges/actual_nil_required_field_Name_nil === RUN Test_Port_CheckChanges/actual_nil_required_field_Network_nil === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set --- PASS: Test_Port_CheckChanges (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_all_required_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Name_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Network_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set (0.00s) === RUN Test_Port_RenderOpenstack === RUN Test_Port_RenderOpenstack/actual_not_nil === RUN Test_Port_RenderOpenstack/actual_nil_success === RUN Test_Port_RenderOpenstack/actual_nil_cloud_error --- PASS: Test_Port_RenderOpenstack (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_not_nil (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_success (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_cloud_error (0.00s) === RUN Test_Port_createOptsFromPortTask === RUN Test_Port_createOptsFromPortTask/all_fields_set === RUN Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups --- PASS: Test_Port_createOptsFromPortTask (0.00s) --- PASS: Test_Port_createOptsFromPortTask/all_fields_set (0.00s) --- PASS: Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstacktasks 0.027s ? k8s.io/kops/upup/pkg/fi/cloudup/spotinsttasks [no test files] === RUN TestWriteValue === RUN TestWriteValue/null === RUN TestWriteValue/empty_list === RUN TestWriteValue/string === RUN TestWriteValue/list === RUN TestWriteValue/list_of_objects === RUN TestWriteValue/object_block --- PASS: TestWriteValue (0.00s) --- PASS: TestWriteValue/null (0.00s) --- PASS: TestWriteValue/empty_list (0.00s) --- PASS: TestWriteValue/string (0.00s) --- PASS: TestWriteValue/list (0.00s) --- PASS: TestWriteValue/list_of_objects (0.00s) --- PASS: TestWriteValue/object_block (0.00s) === RUN TestWriteLiteral === RUN TestWriteLiteral/string === RUN TestWriteLiteral/traversal === RUN TestWriteLiteral/provider_alias === RUN TestWriteLiteral/file --- PASS: TestWriteLiteral (0.00s) --- PASS: TestWriteLiteral/string (0.00s) --- PASS: TestWriteLiteral/traversal (0.00s) --- PASS: TestWriteLiteral/provider_alias (0.00s) --- PASS: TestWriteLiteral/file (0.00s) === RUN TestWriteLiteralList === RUN TestWriteLiteralList/empty_list === RUN TestWriteLiteralList/one_literal === RUN TestWriteLiteralList/two_literals === RUN TestWriteLiteralList/one_traversal_literal,_one_string_literal --- PASS: TestWriteLiteralList (0.00s) --- PASS: TestWriteLiteralList/empty_list (0.00s) --- PASS: TestWriteLiteralList/one_literal (0.00s) --- PASS: TestWriteLiteralList/two_literals (0.00s) --- PASS: TestWriteLiteralList/one_traversal_literal,_one_string_literal (0.00s) === RUN TestWriteMap === RUN TestWriteMap/empty_map === RUN TestWriteMap/simple_map === RUN TestWriteMap/complex_keys === RUN TestWriteMap/literal_list --- PASS: TestWriteMap (0.00s) --- PASS: TestWriteMap/empty_map (0.00s) --- PASS: TestWriteMap/simple_map (0.00s) --- PASS: TestWriteMap/complex_keys (0.00s) --- PASS: TestWriteMap/literal_list (0.00s) === RUN TestWriteMapLiterals === RUN TestWriteMapLiterals/literal_values --- PASS: TestWriteMapLiterals (0.00s) --- PASS: TestWriteMapLiterals/literal_values (0.00s) === RUN TestWriteLocalsOutputs === RUN TestWriteLocalsOutputs/empty_map === RUN TestWriteLocalsOutputs/single_output === RUN TestWriteLocalsOutputs/list_output --- PASS: TestWriteLocalsOutputs (0.00s) --- PASS: TestWriteLocalsOutputs/empty_map (0.00s) --- PASS: TestWriteLocalsOutputs/single_output (0.00s) --- PASS: TestWriteLocalsOutputs/list_output (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraform 0.039s === RUN TestGetOutputs === RUN TestGetOutputs/empty_map === RUN TestGetOutputs/single_output === RUN TestGetOutputs/list_output === RUN TestGetOutputs/duplicate_names === RUN TestGetOutputs/duplicate_values --- PASS: TestGetOutputs (0.00s) --- PASS: TestGetOutputs/empty_map (0.00s) --- PASS: TestGetOutputs/single_output (0.00s) --- PASS: TestGetOutputs/list_output (0.00s) --- PASS: TestGetOutputs/duplicate_names (0.00s) --- PASS: TestGetOutputs/duplicate_values (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraformWriter 0.072s === RUN TestKeypairDeps --- PASS: TestKeypairDeps (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/fitasks 0.041s ? k8s.io/kops/upup/pkg/fi/loader [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/cloudinit [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/local [no test files] === RUN TestArchiveDependencies --- PASS: TestArchiveDependencies (0.00s) === RUN TestBindMountCommands I0920 05:49:05.714126 9619 bindmount.go:224] running mount command [mount --bind /containerized_mounter /containerized_mounter] I0920 05:49:05.714186 9619 bindmount.go:233] running mount command [mount -o remount,exec /containerized_mounter] I0920 05:49:05.714197 9619 bindmount.go:224] running mount command [mount --rbind /var/lib/kubelet/ /containerized_mounter/rootfs/var/lib/kubelet] I0920 05:49:05.714209 9619 bindmount.go:244] running mount command [mount --make-rshared /containerized_mounter/rootfs/var/lib/kubelet] I0920 05:49:05.714227 9619 bindmount.go:224] running mount command [mount --bind -o ro /proc /containerized_mounter/rootfs/proc] I0920 05:49:05.714241 9619 bindmount.go:224] running mount command [mount --bind -o ro /dev /containerized_mounter/rootfs/dev] I0920 05:49:05.714251 9619 bindmount.go:224] running mount command [mount --bind -o ro /etc/resolv.conf /containerized_mounter/rootfs/etc/resolv.conf] --- PASS: TestBindMountCommands (0.00s) === RUN TestBindMountDependencies --- PASS: TestBindMountDependencies (0.00s) === RUN TestFileDependencies === RUN TestFileDependencies/user === RUN TestFileDependencies/parentDir === RUN TestFileDependencies/afterFiles --- PASS: TestFileDependencies (0.00s) --- PASS: TestFileDependencies/user (0.00s) --- PASS: TestFileDependencies/parentDir (0.00s) --- PASS: TestFileDependencies/afterFiles (0.00s) === RUN TestIssueCertFileDependencies --- PASS: TestIssueCertFileDependencies (0.00s) === RUN TestLoadImageTask_Deps --- PASS: TestLoadImageTask_Deps (0.00s) === RUN TestServiceTask_Deps --- PASS: TestServiceTask_Deps (0.00s) === RUN TestServiceTask_UnknownTypes W0920 05:49:05.715875 9619 service.go:92] Unhandled type *nodetasks.FakeTask in Service::GetDependencies: &{} --- PASS: TestServiceTask_UnknownTypes (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/nodeup/nodetasks 0.057s ? k8s.io/kops/upup/pkg/fi/nodeup/nodetasks/dnstasks [no test files] ? k8s.io/kops/upup/pkg/fi/secrets [no test files] === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) === RUN TestStringSlicesEqualIgnoreOrder --- PASS: TestStringSlicesEqualIgnoreOrder (0.00s) === RUN TestHashString --- PASS: TestHashString (0.00s) === RUN TestSanitizeString --- PASS: TestSanitizeString (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/utils 0.034s ? k8s.io/kops/upup/pkg/kutil [no test files] ? k8s.io/kops/upup/tools/generators/fitask [no test files] ? k8s.io/kops/util/pkg/architectures [no test files] === RUN TestFindDistribution --- PASS: TestFindDistribution (0.00s) PASS ok k8s.io/kops/util/pkg/distributions 0.020s ? k8s.io/kops/util/pkg/env [no test files] === RUN TestWithTee --- PASS: TestWithTee (0.00s) PASS ok k8s.io/kops/util/pkg/exec 0.022s === RUN TestNewHasher === RUN TestNewHasher/sha256 === RUN TestNewHasher/sha1 === RUN TestNewHasher/md5 --- PASS: TestNewHasher (0.00s) --- PASS: TestNewHasher/sha256 (0.00s) --- PASS: TestNewHasher/sha1 (0.00s) --- PASS: TestNewHasher/md5 (0.00s) === RUN TestMemberFromString === RUN TestMemberFromString/sha256_1 === RUN TestMemberFromString/sha256_2 === RUN TestMemberFromString/sha256_3 === RUN TestMemberFromString/sha1_1 === RUN TestMemberFromString/sha1_2 === RUN TestMemberFromString/sha1_3 === RUN TestMemberFromString/md5_1 === RUN TestMemberFromString/md5_2 === RUN TestMemberFromString/md5_3 --- PASS: TestMemberFromString (0.00s) --- PASS: TestMemberFromString/sha256_1 (0.00s) --- PASS: TestMemberFromString/sha256_2 (0.00s) --- PASS: TestMemberFromString/sha256_3 (0.00s) --- PASS: TestMemberFromString/sha1_1 (0.00s) --- PASS: TestMemberFromString/sha1_2 (0.00s) --- PASS: TestMemberFromString/sha1_3 (0.00s) --- PASS: TestMemberFromString/md5_1 (0.00s) --- PASS: TestMemberFromString/md5_2 (0.00s) --- PASS: TestMemberFromString/md5_3 (0.00s) === RUN TestFromString === RUN TestFromString/sha256 === RUN TestFromString/sha1 === RUN TestFromString/md5 === RUN TestFromString/unknown --- PASS: TestFromString (0.00s) --- PASS: TestFromString/sha256 (0.00s) --- PASS: TestFromString/sha1 (0.00s) --- PASS: TestFromString/md5 (0.00s) --- PASS: TestFromString/unknown (0.00s) PASS ok k8s.io/kops/util/pkg/hashing 0.024s === RUN TestKeys --- PASS: TestKeys (0.00s) === RUN TestSortedKeys --- PASS: TestSortedKeys (0.00s) PASS ok k8s.io/kops/util/pkg/maps 0.029s ? k8s.io/kops/util/pkg/mirrors [no test files] === RUN TestGetProxyEnvVars W0920 05:49:01.928557 9272 proxy.go:34] EgressProxy set but no proxy host provided --- PASS: TestGetProxyEnvVars (0.00s) PASS ok k8s.io/kops/util/pkg/proxy 0.021s === RUN TestSet === RUN TestSet/simple_setting === RUN TestSet/setting_with_wildcard === RUN TestSet/creating_missing_objects === RUN TestSet/set_int === RUN TestSet/set_int32 === RUN TestSet/set_int64 === RUN TestSet/set_int_pointer === RUN TestSet/set_int32_pointer === RUN TestSet/set_int64_pointer === RUN TestSet/set_enum === RUN TestSet/set_enum_slice === RUN TestSet/append_enum_slice --- PASS: TestSet (0.00s) --- PASS: TestSet/simple_setting (0.00s) --- PASS: TestSet/setting_with_wildcard (0.00s) --- PASS: TestSet/creating_missing_objects (0.00s) --- PASS: TestSet/set_int (0.00s) --- PASS: TestSet/set_int32 (0.00s) --- PASS: TestSet/set_int64 (0.00s) --- PASS: TestSet/set_int_pointer (0.00s) --- PASS: TestSet/set_int32_pointer (0.00s) --- PASS: TestSet/set_int64_pointer (0.00s) --- PASS: TestSet/set_enum (0.00s) --- PASS: TestSet/set_enum_slice (0.00s) --- PASS: TestSet/append_enum_slice (0.00s) === RUN TestSetInvalidPath === RUN TestSetInvalidPath/setting_with_wildcard === RUN TestSetInvalidPath/creating_missing_objects === RUN TestSetInvalidPath/set_int === RUN TestSetInvalidPath/set_int32 === RUN TestSetInvalidPath/set_int64 --- PASS: TestSetInvalidPath (0.00s) --- PASS: TestSetInvalidPath/setting_with_wildcard (0.00s) --- PASS: TestSetInvalidPath/creating_missing_objects (0.00s) --- PASS: TestSetInvalidPath/set_int (0.00s) --- PASS: TestSetInvalidPath/set_int32 (0.00s) --- PASS: TestSetInvalidPath/set_int64 (0.00s) === RUN TestUnset === RUN TestUnset/simple_unsetting === RUN TestUnset/unsetting_with_wildcard === RUN TestUnset/uset_int === RUN TestUnset/unset_int32 === RUN TestUnset/unset_int64 === RUN TestUnset/unset_int_pointer === RUN TestUnset/unset_int32_pointer === RUN TestUnset/unset_int64_pointer === RUN TestUnset/unset_enum === RUN TestUnset/unset_enum_slice --- PASS: TestUnset (0.00s) --- PASS: TestUnset/simple_unsetting (0.00s) --- PASS: TestUnset/unsetting_with_wildcard (0.00s) --- PASS: TestUnset/uset_int (0.00s) --- PASS: TestUnset/unset_int32 (0.00s) --- PASS: TestUnset/unset_int64 (0.00s) --- PASS: TestUnset/unset_int_pointer (0.00s) --- PASS: TestUnset/unset_int32_pointer (0.00s) --- PASS: TestUnset/unset_int64_pointer (0.00s) --- PASS: TestUnset/unset_enum (0.00s) --- PASS: TestUnset/unset_enum_slice (0.00s) === RUN TestUnsetInvalidPath === RUN TestUnsetInvalidPath/usetting_with_wildcard === RUN TestUnsetInvalidPath/missing_objects --- PASS: TestUnsetInvalidPath (0.00s) --- PASS: TestUnsetInvalidPath/usetting_with_wildcard (0.00s) --- PASS: TestUnsetInvalidPath/missing_objects (0.00s) === RUN TestParseFieldPath === RUN TestParseFieldPath/test_Spec.Containers.Image === RUN TestParseFieldPath/test_Spec.Containers[0].Image === RUN TestParseFieldPath/test_Spec.Containers[*].Image --- PASS: TestParseFieldPath (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers.Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[0].Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[*].Image (0.00s) PASS ok k8s.io/kops/util/pkg/reflectutils 0.011s === RUN TestGetUniqueStrings --- PASS: TestGetUniqueStrings (0.00s) === RUN TestContains --- PASS: TestContains (0.00s) PASS ok k8s.io/kops/util/pkg/slice 0.009s ? k8s.io/kops/util/pkg/tables [no test files] === RUN TestSplitToSections --- PASS: TestSplitToSections (0.00s) PASS ok k8s.io/kops/util/pkg/text 0.042s ? k8s.io/kops/util/pkg/ui [no test files] === RUN TestAzureBlobPathBase === RUN TestAzureBlobPathBase/Test_case_0 === RUN TestAzureBlobPathBase/Test_case_1 === RUN TestAzureBlobPathBase/Test_case_2 --- PASS: TestAzureBlobPathBase (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_2 (0.00s) === RUN TestAzureBlobPathPath === RUN TestAzureBlobPathPath/Test_case_0 === RUN TestAzureBlobPathPath/Test_case_1 === RUN TestAzureBlobPathPath/Test_case_2 --- PASS: TestAzureBlobPathPath (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_2 (0.00s) === RUN TestAzureBlobPathJoin --- PASS: TestAzureBlobPathJoin (0.00s) === RUN TestBuildVaultPath context_test.go:27: No vault dev token set --- SKIP: TestBuildVaultPath (0.00s) === RUN TestCreateFile --- PASS: TestCreateFile (0.00s) === RUN TestWriteTo --- PASS: TestWriteTo (0.00s) === RUN TestMemFsCreateFile --- PASS: TestMemFsCreateFile (0.00s) === RUN TestMemFsReadDir --- PASS: TestMemFsReadDir (0.00s) === RUN TestMemFsReadTree --- PASS: TestMemFsReadTree (0.00s) === RUN Test_VFSPath --- PASS: Test_VFSPath (0.00s) === RUN Test_S3Path_Parse --- PASS: Test_S3Path_Parse (0.00s) === RUN Test_newVaultPath vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_newVaultPath (0.00s) === RUN Test_newVaultPathHostOnly --- PASS: Test_newVaultPathHostOnly (0.00s) === RUN Test_WriteFileReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileReadFile (0.00s) === RUN Test_WriteFileDeleteFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileDeleteFile (0.00s) === RUN Test_CreateFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_CreateFile (0.00s) === RUN Test_ReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadFile (0.00s) === RUN Test_Join vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_Join (0.00s) === RUN Test_VaultReadDirList vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadDirList (0.00s) === RUN Test_ReadDir vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadDir (0.00s) === RUN Test_IsDirectory vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_IsDirectory (0.00s) === RUN Test_VaultReadTree vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadTree (0.00s) PASS ok k8s.io/kops/util/pkg/vfs 0.019s === RUN TestGSRenderTerraform === RUN TestGSRenderTerraform/gs://foo/bar --- PASS: TestGSRenderTerraform (0.00s) --- PASS: TestGSRenderTerraform/gs://foo/bar (0.00s) === RUN TestS3RenderTerraform === RUN TestS3RenderTerraform/s3://foo/bar W0920 05:49:06.743661 9754 s3context.go:239] Unable to read bucket encryption policy for "foo" in region "us-east-1": will encrypt using AES256 --- PASS: TestS3RenderTerraform (0.00s) --- PASS: TestS3RenderTerraform/s3://foo/bar (0.00s) PASS ok k8s.io/kops/util/pkg/vfs/tests 0.067s >>> kops: Entering fakeroot... >>> kops*: Running postcheck for kops >>> kops*: Preparing package kops... >>> kops*: Stripping binaries >>> kops*: Scanning shared objects >>> kops*: Tracing dependencies... >>> kops*: Package size: 128.4 MB >>> kops*: Compressing data... >>> kops*: Create checksum... >>> kops*: Create kops-1.25.0-r0.apk >>> kops: Build complete at Tue, 20 Sep 2022 05:49:53 +0000 elapsed time 0h 3m 16s >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Uninstalling dependencies... (1/3) Purging .makedepends-kops (20220920.054639) (2/3) Purging kubernetes (1.25.0-r1) (3/3) Purging go (1.19.1-r0) Executing busybox-1.35.0-r25.trigger OK: 398 MiB in 98 packages >>> kops: Updating the testing/x86_64 repository index... >>> kops: Signing the index...